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ABSTRACT 

Reported is a compilation of the data collected from 
a comprehensive evaluation of the Information Retrieval Center of the 
Northern Colorado Educational Board of cooperative Services (NCEBOCS) 
The evaluation consists of an historical review of the Center 
(1 967—1970), analysis of requests received, iiser reactions to the 
services. Interviews with administrative personnel of the member 
school districts, interviews with coordinators from each of the seven 
participating states, interviews with Retrieval center staff members, 
evaluation of a two-week workshop attended by key users, suggestions 
from teacher trainers on the operation of the Center, and progress 
reports provided to the Retrieval Staff on use in improving the 
operation of the Center. The evaluators consider the Information 
Retrieval Center an effective unit. Response to the service is 
favorable, and the center appears to have some impact on educational 
practice. Recommendations for change relate primarily to rt;f ineir.ent 
and extension of present activities. Data compiled is appended in 
tabular form. (SJ) 
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Preface 



In July, 1970, a contract was defined wherein the Human Factors 
Research Laboratory at Colorado State University would conduct a compre- 
hensive evaluation of the Information Retrieval Center of the Northern 
Colorado Educational Board of Cooperative Services. The essential plan 
for the evaluation provided that a variety of sources for evaluation data 
would be considered and several kinds of information would be collected. 

A copy of the evaluation proposal is shown in Appendix A. Review of the 
proposal indicates that specific details of the evaluation were to be 
developed later, however, and some of the evaluation would depend upon the 
activities in which the Retrieval Center was engaging at the time the 
evaluation was made. As the Fiscal Year 1970-71 progressed, details of 
the evaluation were finalized and appropriate data collected. In its final 
format, the evaluation consisted of the following: 

1. An historical review of the Center from its inception in 1967 
through 1970 was developed. This review incorporated all historical 
data available in the files of the Center. 

2. All requests received by the Center between its inception In 1967 
through December, 1970 were classified into topical categories, 
coded, and analyzed according to thirteen variables, 

3. A comprehensive user questionnaire was developed to assess 
reactions of users to the Center. This questionnaire was mailed 

to a 20% sample of users during calendar year 1970. More than 
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15 % of those receiving the questionnai re responded. Reactions of 
the user sample were coded and analyzed according to three 
demographic characteristics of the users. 

4. Personal interviews were conducted with at least two admini- 
strative personnel from each of the seven BOCES districts served 
by the Center in the Fall of 1970, and at least one interview with 
the administrators of each district was conducted during the Spring 
of 1971. Interview responses were summarized and interpreted, 

5. Coordinators from each of the states participating in Information 
Retrieval Center activities (Colorado, Wyoming, Utah, Nevada, Wash- 
ington, Oregon, and South Dakota) were interviewed in depth and 
complete interview reports were prepared for each state. The 
interview reports included observations of the interviewers and 

data obtained when the states were visited. 

6. Staff members of the Retrieval Center were interviewed for their 
reactions and suggestions regarding operation of the Center. These 
interviews were summarized and converted to operational suggestions. 

7. An evaluation was conducted of a two-week institute attended by 
key personnel from participating states and districts. This evaluation 
was disseminated to Retrieval staff personnel as one portion of an 
on-going assessment of Retrieval Center procedures. 

8. Interviews were also conducted with teacher trainers on university 
campuses to obtain their suggestions about the operation of the 




Retrieval Center. As with other interview data, responses were 
summarized and translated into operational suggestions, 

9, Several verbal and written progress reports were provided to 
the Retrieval Center staff throughout the year for use in improving 
the operation of the Center. In some instances, these reports were 
incorporated into evaluations being made by other agencies such as 
the Colorado State Department of Education. 

The present volume is a compilation of the data collected through 
the foregoing efforts. The report is organized according to the evaluation 
activities as outlined and can be studied in total or in segments as 
the purpose of the reader dictates. 
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I, Historical Review 
of 

The Information Retrieval Center of the NCBOCES 



Introductio n 

On November 16, 1966, the Boulder Valley School District RE-2J received 
a planning grant from the U,S. Office of Education under the Elementary 
and Secondary Education Act Title III for the development of a "Cooperative 
Community Educational Resources Center." The project was initially funded 
from January 1, 1967 to December 31, 1967. Additional grants have been 
received annually which permitted project continuation. Initially, the 
project was under the directorship of Mrs. Violet Wagener (1967-63) and 
subsequently Mr. William H, McCleary (1969-71). 

On May 1, 1970, the Resources Center was incorporated under the 
Northern Colorado Board of Cooperative Education Services (NCBOCES), The 
Educational Information Retrieval Service has continued to be one of the 
functions performed by the BOCES. Other activities have included program 
development and program evaluation (through 1970). 

The original proposal addressed itself to the increasing need for 
bringing the consumers of educational resources closer to research 
and resource information in the field of education. In addition, the need 
was expressed for this information to be provided in a more efficient and 
effective manner. The original purpose of the Resources Center, then was 
the establishment of a communications process which would provide educators, 
administrators and the community with information on the latest developments 

in the field of education. 
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As stated in the 1970 Continuation Grant Proposal* the general 
objectives were as follows* 

"i. To keep individual educators aware of proresslonal developments 
within specific areas of interest and abreast of the availability of the 
research and reference materials supporting those needs and interests, 

"2. To provide educators with the opportunity to make formal requests 
for research materials in support of either specific curriculum areas or 
broad educational topics of interest, 

"3, To provide educators with the knowledge of a wider variety of 
appropriate instructional resources for individualizing the learning process. 
Service may be obtained on a routine current awareness or single request 
basis . " 

In attempting to fulfill these objectives* the Resources Center has been 
operating In the following manner: The Center provides educational personnel 

contacting it (teachers, librarians, teacher aides, administrators, and 
parent study groups) with abstracts and summaries of educational data in 
the form of computer printouts. In addition, the Center provides manual 
search services by reference librarians who provide annotated lists and 
summaries of print and non-print materials and resources requested by 
people using the Center's services. Individuals with a problem of an 
educational nature or area of Interest may contact the Information Retrieval 
Center by mail or phone. Members of the staff analyze the request and 
assign educational key words or descriptor terms matching the request to 
the problem area. While these terms are fed into the computer and abstracts 
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from the stored data bank obtained, reference librarians conduct a manual 
search of local libraries and reference centers. The result of this 
unique combination of in-depth computer search and manual search is a 
profile of recent research reports, dissertation summaries, periodicals, 
books, and other pertinent information which is compiled and mailed to the 
requestor. 

Each us°r : s request is maintained in a log containing the nature of 
the request, the time necessary to process the request, the results of the 
search in terms of the quantity of abstracts and manual search articles 
in the profile, and Information about possible follow-up requests by the 
user. 

After receiving the profile of educational Information relevant to 
his request, the requestor may order from the Center full documents of 
abstracts he would like to study further in microfiche or hard copy. 

He returns any orders he might have along with an evaluation form indicating 
the usefulness of the materials he had been sent. The user may check out 
microfiche for a 2-week period. Xerox copies and/or future articles may 
be obtained from the reference librarians to fill follow-up requests as 
indicated on the aforementioned evaluation forms. The source of most of 
the information in the computer data base was abstracts prepared by 
clearinghouses of the U,S. Office of Education's Educational Resources 
Information Center (ERIC). 

In addition to Colorado, five other states participated in the 
educational information retrieval services of the Boulder Center in 
1970. South Dakota had a direct communi cations line to the Center and 
requests were made directly. Utah, Oregon, Washington, and Wyoming 
established various communications networks of their own. Nevada 
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participated informally as did requestors from several other states. 

Although the program originally leased processing time on a CDC 3800 
computer* it later leased a UN I VAC 9300 computer housed in the Center. 

The data base was divided into two files, one a Current Interest and one 
History file. The Current Interest file consisted of not more than 5,000 
abstracts containing only the most current information in representative 
educational topic areas. It was used for preliminary or initial infor- 
mation retrieval requests and for periodic current interest requests. 

The History file contained all other abstracts. 



Summary of 1967 Planning Grant 

The activities of the 1967 period included attempts to establish a 
data base, the development of a computer printout system* a manual search 
service, a current awareness service, and the adaptation and evaluation 
of two generalized Information retrieval programs. 

One concern was the development of a plan for selective dissemination 
of information to staff members in the Boulder County School District 
and examination of the plan on a pilot basis. IBM Boulder Education 
Department representatives were consulted and relevant literature was 
searched to determine the feasibility of using the IBM SDI system for 
the purpose of the project. 

Trial runs of a selective dissemination service were made utilizing 
the BIRS (Basic Indexing Retrieval System) and INFOL (Information Oriented 
Language) programs. It was decided that the INFOL program was very 
sensitive to keypunch errors but allowed more data manipulation and was 

selected as the project's information retrieval program. Research In 
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Education abstracts, locally abstracted materials including non-print 
resources. University of Colorado Education Library book acquisitions 
and project library materials were keypunched and became the core for the 
f rst data base. 



To facilitate the search with descriptors, a word frequency list. 



KWOC (Keyword out of context) list and descriptor print-out were added 
to the ERIC thesaurus. Instead of categorizing literature by level of 
comprehension difficulty, standard categories with terms such as elementary, 
secondary, administration, teaching, etc,, were assigned to all profiles, 

A pilot-operational selective dissemination service was designed, 
and it was anticipated that it would provide a more efficient and effective 
way of handling the acquisitioning, cataloguing, and processing of 
materials. Storage capacity would be large, and it was considered feasible 
that an indirect service of regional processing would emct ge. 

While this program was in operation, work progressed on dissemination 
at the state and national levels. At the local level, schools and school 
districts interested in receiving the services of the Community Resources 



Center were given formal presentations during faculty and staff meetings 
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in the 1967 school year. 

Seventy-four interest profiles were written for professional educators 
and administrators at the local district level. Individuals served outside 
the district included members of the Colorado Department of Education, 
University of Colorado School of Education, University of Denver School of 
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Education, Adams County District 12, Adams County District 50, Jefferson 
County School District, Social Sciences Consortium, University of Colorado, 
and Title III Directors in Colorado. Computer printouts were distributed 
to the trial users with an evaluation sheet accompanying each profile. Current 
awareness profiles were rewritten as interests and descriptors changed in 
an effort to increase the relevancy of abstracts to user i nterest. The 
current awareness searches were usually less specific than the information 
retrieval or retrospective searches. Manual searches frequently accompanied 
the computer printouts of abstracts. Both manual and information retrieval 
searches, for educators outside Boulder Valley District, increased as the 
service of the project became known even though dissemination outside the 
district was minimal. 
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Summary of 1968 Pilot-Operational Grant 
OE Project No, OEG 8-8-004438-0019 (056) 

January 1 , 1968 to December 31, 1968 

Objectives of the 1968 Pilot-Operational Grant were the following: 

1. To keep teachers and administrators aware of developments and 
educational resources in the educational field by providing a dissemination 
of information service. 

2. To provide knowledge of a wider variety of appropriate 
Instructional resources for individualizing the learning process, 

3. To investigate random access to instructional resources via 
computer systems, 

4. To provide an educational learning avenue for secondary students 
to make their own identification of resources found in their community 
envi ronment. 

5. To provide information in abstract form about current educational 
literature to the community through the cooperating agencies and school 
related groups. 

Dissemination of the project will be directed to four audiences: 
Teachers, students and cooperating agencies in their roles of users 
and suppliers of information; the Boulder County community In its role 
of a resource and potential user; interested persons in education, library 
and information science fields; educational agencies outside Boulder 
County who are potential cooperating agencies. 

Among the activities and procedures utilized for accomplishing the 
project objectives were the selecting, abstracting and describing of 
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literature in the educational ■field. All abstracts and descriptors applicable 
to the needs of the elementary, secondary, and junior college education 
from the 1966 and 1967 Issues of Rese a rch in Education were added to 
the Community Resources Center's data base. Additionally, the project 
abstractor wrote, descriptorized and coded abstracts of new books, papers 
and articles which the project acquisitioned. The abstracting of profes- 
sional materials, particularly pamphlets, non-print and curriculum 
materials received by the Center continued to grow. Descriptors were 
assigned to books acquisitioned by the University of Colorado Education 
Library and complete Library of Congressional bibliographic data was 
included in the input to the system. Annotations, summaries and brief 
abstracts were added to the data base when appropriate. ERIC abstracts 
and locally written abstracts continued to be punched for both the BIRS and 
INFOL systems. In early September, keypunching of ERIC abstracts was 
decreased because the amount of input in the data base was considered 
sufficient for utilization during the pilot phase of the project. It 
was anticipated that the ERIC magnetic tapes would be available in July 
1969 and would expand the data base. 

Dissemination of this information routinely to users according to 
their individual Interests and needs was conducted through the Information 
Retrieval system. The number of requests for searches Increased con- 
siderably, and information retrieval became more efficient as descriptors 
were applied more accurately. However manual checking was conducted on 
all print-outs and profiles as a form of final profile evaluation. 

Feedback from the users was further used to increase the accuracy of 
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match between their interests and needs and the abstracts and descriptions 
sent to them. Generic profiles based on commonality of interests were 
gradually compiled. 

Use of the BIRS computer system was discontinued in preference to 
the INFOL system. Two of the major problems encountered with the 
BIRS system were the necessity to code descriptors manually and the 
limitation of types of retrieval. Despite the exactness and precision 
required by INFOL, its versatility, format, and lack of need for manual 
work made it a better tool for the project. 

Instructional materials were selected by teachers, librarians and 
supervisors for use in the overall school curriculum. Descriptors of 
these materials, plus the Community Educational Resources and the 
field trip files, were added to the data base started the previous year. 
Activities of the project were to develop concurrently and in the 
following relationship. 
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The Center's activities expanded in two directions. This included 
closer coordination with the Boulder Valley District Instructional 
Materials Center and more service to educators outside the school 
district. Experimentation with new techniques for input selection 
and abstraction as well as new techniques for automated dissemination 
of information were expected to be continued under the new grant. 

On January 2, 1969, Mr. William H. McClea . „ was hired as Project 
Director, With the expansion of funding and services, the 3- member 
staff was increased to 10, 

The manual search aspects of the project continued with the 
Reference Librarian providing information when no elements were available 
from the computer. 

The community resources and human resources files were expanded 
and continued to gain information researched on businesses, individuals and 
government services in the local area. 
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Summary of 1969 Operational Grant 
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The Operational Grant for the Cooperative Community Educational 
Resources Center was funded for the period January 1, 1969 to December 30, 
1969. The project, under this grant, was a continuation of planning 
and procedures established in the Pilot-Operational Grant of 1968, 

Two general objectives of the project were stated in the proposal: 

"(1) To keep educators aware of professional development and resources 
in the educational field by providing dissemination of current information 
according to needs and interests of users. (2) To provide educators 
with knowledge of the variety of Instructional resources for individualizing 
the learning process. This Current Awareness service can be obtained on 
a routine or one-time basis," 

The specific objectives of the project and the procedures for carrying 
them out can be arranged in four major categories of activities: On-going 

operations, expansion, experimentation and development, and evaluation. 

On-going Operations 

The main operations of the Center were the Current Awareness 
and Information Retrieval services. These were continued as adjusted and 
approved in previous grants and expanded to other users. Current Awareness 
users were to receive 10 monthly abstracts relating to their needs as 
indicated on an Interest profile. However, the Current Awareness aspect 
of the Center was not fully developed. Reasons for this included 
the delay in obtaining ERIC t.pes to fully expand 1:he data bank, the 
excessive cost of retrieving stored information under the system used 
(CDC 3800 computer), and the emphasis in the project upon giving individual 
research and reference assistance to users. The Information Retrieval 
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System of the 1969 Grant was conducted as described in the first part 
of this chapter. The goal was to return requested information to the 
user within a period of 72 hours. 

Computer and manual searches, as developed in previous grants, 
were continued, in order for abstractor and reference librarians to 
effectively Interpret the user's question, it was considered necessary 
that they be familiar with the descriptor terminology used in the 
computer. Continuing emphasis was on the refinement of the descriptors 
and their use. 

Information Retrieval services were available to users within the 
Boulder Valley School District and other schools, districts, and 
educational agencies, both in Colorado and other States. The Center 
obtained complete documents upon request for users within the Boulder 
Valley School District only. Requestors outside the District were 
encouraged to make contact with the Center through their curriculum direc- 
tor or the librarian in their district. Users contacted the Center 
j through letter, telephone or in person. 

Another on-going activity at the Center was the dissemination of 
j information about its use. The Project Director gave project presentations 
s to faculties acquainting them with the Center's services. At the end of 

f _ 

f this grant, 65% of the schools in Boulder Valley had been given a presen-, 

,v 

tation. A newsletter about the project was expanded to include district 
l librarians* and educational mediasts in each school who were designated as 

r * , 

I the project contact agents. A workshop was held in January 1969 for these 
| contact agents. Dissemination was also accomplished through state-wide 
t professional organizations, 

! o 
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Expansion 

The Information Retrieval service was constantly being expanded to 
Include other users through the methods of dissemination described above. 
The maximum goal set for the Center in the 1969 proposal was to handle an 
average of 350 profile requests per month for 10 months and unlimited 
manual search services. Boulder Valley teachers were to obtain up to 125 
profiles per month while other Colorado users were not expected to request 
service as often. 

The data base was to be constantly expanded through addition of 
abstracts from ERIC tapes and local sources abstracted at the Center. 
Criteria for selection of materials to be put into the data base included 
the availability of the source to the user. 

Further development of the logic and listing capabilities of the 
INFOL 3000 series was carried out. However, the CDC 3800 computer, on 
which this system was based, was not considered completely satisfactory 
and other systems were investigated. The decision was made to lease a 
UNIVAC 9300 computer. 

Experimentation and Development 

This category included a variety of activities for improving and 
finding new uses for the services of the Resources Center. Of major 
importance was the continued development of descriptor terms and subject 
headings which accurately and precisely reported the content of the 
material. Concentration was also focused on developing new uses of the 
Current Awareness and Information Retrieval service for professional 
growth. Ideas for this included using the services as a source for articles 
in newsletters, for use of curriculum committees and for research design. 
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The use of the generic profiles was further developed. These 
profiles indicated the most commonly requested educational information 
reflecting the interests and concerns of teachers. The profiles could 
be used for newsletters, audio-tape programs and other activities to 
encourage professional growth. 

Development of the Center's technical areas was attempted through 
investigation of other information retrieval centers for ideas which 
could be adapted. 

Eval uati on 

Continuous on-going evaluation of the system was conducted by the 
staff. User-returned evaluation forms were sent out with each completed 
profile to assess the usefulness of materials. The Director made spot- 
checks of computer print-out profiles to insure that the materials were 
relevant to the requestors' needs. Interviews were conducted in October, 
1968 by the team of Olson, Donohue, Lennox, Schmidt and Seager; another 
was conducted by the firm of Rouche ' , Ross, Bailey, and Smart in April, 
1969, For details consult the 1970 Continuation Proposal, 
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Sunrnary of 1970 Continuation Grant 

The Continuation Grant for the Community Education Resources Center 
was funded for the period beginning March 1, 1970 and ending June 30, 1971, 
This project was a continuation of the previous Operational Grant with a 
few changes as noted below. On May 1, 1970, the Resources Center was 
incorporated under the Northern Colorado Board of Cooperative Educational 
Services (NCBOCES) « 

Three general objectives were recorded in the 1970 proposal, two 
being the same as the objectives stated in the 1969 proposal. The third 
objective was to provide educators with the opportunity to make requests 
for research materials related to specific curriculum areas or broad 
educational topics of interest. This was simply a further explanation of 
the basic project objective. 

The specific objectives and procedures mentioned in the 1970 Contin- 
uation Grant were the same as those of the 1969 grant except for the 
following changes: 

1, The turn-around time for returning profiles to users was to be 
changed from 72 hours to oni week. 

2, Greater emphasis was to be placed on refinement in the use of 
descriptor terms and abstract writing to provide more specific and 
detailed information to the user. 

3, An additional evaluation of the project was conducted in November, 
1969, by the team of Broderius, Stager, Yorke, and Zimmer, 

4, The INF0L program used on the CDC 3800 computer was not considered 
satisfactory and a UNIVAC 9300 computer was to be leased for the 1970 
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Continuation Grant. A discussion of the reasons for this change can be 
found in the 1970 Proposal for Continuation Grant (pp, 22-23) and a 
description of the UN I VAC 9300 system in Appendix E of that same report. 

The Resources Center constantly expanded its service to new 
users during the 1969-70 period. The greatest concentration of users 
was in Boulder County, but other counties in Colorado also employed the 
service frequently. At the time of the writing of the 1970 Continuation 
Proposal, the Center had made eight official and unofficial contracts 
and agreements for use of the Center with the educations ' agencies outside 
Col orado. 



The foregoing sections sections describe the historical development of the 
Retrieval Center up to the time of the Human Factors Research Laboratory 
evaluation in July, 1970. The functioning of the Center during 1970-71 
is described in the subsequent sections of this report. 



II . Description and Analysis of Requests 



To determine the types of requests for information forwarded to the 
Retrieval Center, the file for each request received since the Center's 
inception was consulted and the following data were recorded: Name of 

user, address, state or district in which the user lived, occupation or 
title of user, institution with which the user was affiliated, subject 
of request, date of request, processing time, type of information forwarded 
to user, reactions of user to the information, and additional comments 
on the user evaluation form return to the Center, This information was 
obtained on 2,977 requests. The data for each request were then coded 
according to the code sheet shown in Appendix B, Inspection of the coding 
key reflects the nature of the variables on which the requests were to 
be analyzed. Two classifications for subject of request were used, the 
first a classification designed by the Project Director consisting of 
eight categories, and the second a classification suggested by the ERIC 
Centers established throughout the United States. All subject classifi- 
cations were made by the Project Director, 

In Table 1 are shown the requests classified by state or district. 
Inspection of this Table reveals that the largest number of requests 
was made by personnel in the Bouleer Valley Public Schools and the second 
largest number by personnel in Colorado, not residing in the Northern 
Colorado B0CE5. The largest number of requests from states outside 
Colorado included Washington and South Dakota. 
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Table 1 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Requests by State or District 



No. of 



State or District 



1,018 


34.96 


1. 


Boulder Valley Public Schools RE-2J 


19 


0.65 


2. 


Westminster, Adams 50 School District 


98 


3.37 


3. 


Thornton-Northglenn, Eastlake, Adams 12 


16 


0.55 


4. 


Loveland (Big Thompson), Larimer R-2J 


63 


2.16 


5. 


Fort Collins, Poudre, Larimer R-l 


51 


1.75 


6. 


Longmont, St. Vrain Valley, Boulder RE-1J 


16 


0.55 


7. 


Estes Park, Larimer R-3 


457 


15.68 


8. 


Colorado, not in NCB0CES 


50 


CM 

t^s 

• 


9, 


Nevada 


311 


10.68 


10. 


South Dakota 


354 


12.16 


11. 


Washington 


62 


2.13 


12. 


Oregon 


225 


7.73 


13* 


Utah 


Cl 

I 


1.75 


14. 


Wyomi ng 


121 


4.16 


15. 


All other states 


2,912 


100.00 




Total 
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In Table 2 are shown the occupations or titles of the requf ^ors. 

Here it can be seen that the largest number of requests came from 
specialists, coordinators, directors and consultants. As a group these 
individuals might be classified as change agents in their various 
professional capacities. Teachers at the elementary, secondary and 
college level reflected the next largest number of requestors, followed 
by administrators as a group. It is conceivable that these two categories 
of requestors represent underestimates because of having submitted their 
requests through various specialists. It is apparent, however, regardless 
of the possible underestimation, that primary use of the Center was by 
educational practitioners rather than by individuals outside school 
positions. It should be noted that Table 2 is based on requests rather 
than users per se and that one user may have made more than one request. 

The type of institution with which requestors are identified is 
shown in Table 3, District Administrative Units represented the largest 
category of requestors followed by junior high and high school combined. 

In Table 4 are shown the number of requests by date of request in 
six month intervals. From this table the steady, albeit rapid, growth 
of the use of the Center is apparent. A pronounced increase in use of 
the Center can be seen during 1969 and 1970. 

Processing time for requests is also shown in Table 4, The mean 
processing time for the computer searches is somewhat less than that for 
the manual searches. 

Table 5 contains the number of requests by type of profile issued 
to requestors. The effectiveness of the computer is reflected throughout 
9 the entries in the table. 
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Table 2 

Evaluation of the Information Retrieval Center, Boulder, Colorado 



Number of Requests by Occupation or Title 



No* of 

Requests % Occupation or Title 



114 


4.30 


1 . 


199 


7.51 


2. 


321 


i; 10 


3. 


98 


3,70 


4. 


263 


9.91 


5, 


1,033 


39.00 


6, 


95 


3.57 


7. 


233 


8.79 


8. 


194 


7,30 


9. 


20 


0.76 


10. 


52 


1.96 


11, 


27 


1.10 


12. 


2,649 


100.00 





Student 

Teacher, Elementary 
Teacher, Secondary 
Teacher, College or University 
Librarian 

Specialists, Coordinators, Directors 
and Consultants 

Pupil Personnel — Counselor, 
Psychologist, School Nurse 

Ach - ilstrator - Principal 

Administrator - Superintendent 

Layman 

Other 

Teacher - Special Education 
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Table 3 

Evaluation of the Information Retrieval Center* Boulder, Colorado 
Number of Requests by Type of Institution 

No. of 



Requests 


% 


Type of Institution 


467 


16.39 


1 . 


Elementary School 


682 


23.95 


2. 


High School, Jr. High 


294 


10.32 


3. 


College or University 


233 


8.19 


4. 


State Department of Education 


82 


2.88 


5. 


Research Organization 


791 


27.77 


6. 


District Administrative Unit 


7 


0.25 


7. 


Teacher Training Institution 


193 


6.78 


8. 


ERIC and other Resource Centers 


83 


2.91 


9. 


Other 


16 


0.56 


10. 


Special Education Institution 
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Table 4 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Requests by Date of Request 



No, of 

Requests Date of Request 



4 ( 0.14%) 
86 ( 2.95%) 
77 ( 2.64%) 
227 ( 7.79%) 
459 (15.76%) 
502 (17.23%) 
1,558 (53.49%) 

2,913 



Prior to 1968 
January to June, 1968 
July to December, 1968 
January to June, 1969 
July to December, 1969 
January to June, 1970 
July to December, 1970 



Mean Processing Time 



Mean Computer Processing Time (N=2158) 
(Standard Deviation) 

Mean Manual Processing Time (N-1012) 
(Standard Deviation! 



18.48 days 
(11,470) 

20.56 days 
(16.741) 



£ 



o 

ERIC 
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Table 5 

Evaluation of the Information Retrieval Center,, Boulder, Colorado 



Number of Requests Per Type of Profile Sent 



No. of 

Requests Type of Profile 



994 (36.91%) 
46 ( 1.71%) 
365 (13.55%) 
414 (15,37%) 
638 (23,69%) 
9 ( 0.33%) 
227 ( 8 . 44 %) 
2693 



Current Interest only (Computer) 

History only (Computer) 

Manual Search only 

Current Interest and History only (Computer) 
Current Interest and Manual Search 
History and Manual Search 
Current Interest, History and Manual Search 



F 

! 

\ 

); 



\ o 

ERIC 



\ 

\ 
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Reactions to materials forwarded to the requestors by the Center is 
shown in Tables 6-8, Whereas many evaluation forms were not returned by 
the users, it can be seen that the majority of the reactions were favorable 
to the Center, More than half of the respondents indicated that the 
materials were pertinent and that they would be willing to pay for services 
on a cost-per-profile basis. 

In Table 9 are shown the number of request? according to the Human 
Factors Research Laboratory (HFRL) subject code. Most popular subject of 
requests was the Administrative Procedures and Organization category. 

This category included such requests as. open scheduling, merit pay* contract 
negotiations and school finance. The category, "Instructional Procedures 
and Techniques," was the second most popular. This category included such 
topics as the project method of teaching, computer assisted instruction, 
and pupil participation in learning. Inspection of Table 9 reflects the 
nature of the jobs held by the requestors, shown earlier. 

In Table 10, the requests have been categorized according to the ERIC 
subject code. Here it can be seen that administration again ranks high 
followed by Library and Information Services and Counseling and Student 
Personnel Services, In the ERIC subject code, requests are classified 
according to the specific content of subject matter, a factor not shown 
in the HFRL classification. Cross classifications of requests by the 
ERIC subject code Reading, ERIC subject code Educational Administration, 
and HFRL subject code Curriculum Development, HFRL subject code Instructional 
Procedures and Techniques, and HFRL Administrative Procedures and 

| Organization are shown in Appendix C. 

f 

3 
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Table 6 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

Number of Requests Per Answer to Question #1 
on Evaluation Form 

Were the abstracts in this profile pertinent to your request? 

No, of 

Re quests Answer 



65 


(1 7.57%) 


100% 


140 


(37.84%) 


75% 


48 


(12.97%) 


50% 


101 


(27.30%) 


25% 


16 


( 4.32%) 


0% 



O 

ERJC 

34 
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Table 7 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

Number of Requests Per Answer to Question #2 
on Evaluation Form 

Were the Manual search materials in the profile pertinent? 



No. of 

Requests Answer 

86 (42.57%) 100% 

62 (30.69%) 75% 

13 ( 6.44%) 50% 

28 (13.86%) 25% 

13 ( 6.44%) 0% 
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Table 8 

Evaluation of the Information Retrieval" Center, Boulder, 



Number of Requests Per Answer l Question #3 
on Evaluation Form 

Would you be willing to pay for educational information 
services on a Cost- Per- Profile basis? 



No , Of 

Requests Answer 



307 (87.71%) Yes 

_43 (12,29%) No 

350 



AdditionaT Comments on Evaluation Form 



ERIC 



No. Of 
Requests 

147 (61.51! 
30 (12.55 
35 (14.64%) 
28 (11.71%) 
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Comment 

Generally positive 
Positive with reservations 
Negative 
Neutral 



Colorado 



retri eval 
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Table 9 

Evaluation of the Information Retrieval Center* Boulder, Colorado 
Number of Requests Per HFRL Subject Code 



No, of 

Regues ts % Subject of Requests 



399 


13.46 


1 . 


Curriculum Development 


340 


11.47 


2. 


Pupil Characteristics, Behavior, Guidance 


618 


20.84 


3. 


Instructional Procedure and Techniques 


833 


28.09 


4. 


Administrative Procedure and Organization 


106 


3.58 


5, 


Teacher education, pre and in-service 


273 


9.21 


6. 


Subject matter topic 


134 


4.52 


7. 


Special Education 


262 


8.83 


8, 


Instructional Resources, Research 


0 

2365 


0.00 

100,00 


9. 


Other 



Table 10 29 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

Humber of Requests Per ERIC Subject Code 



Ho, of 



d 

ERJC 



7 


0.24 


1 . 


245 


8.26 


2. 


50 


1.69 


3. 


47 


1.59 


4. 


978 


32.98 


5, 


105 


3,54 


6, 


123 


4.15 


7. 


21 


0.71 


8. 


9 


0.30 


9, 


255 


8.61 


10, 


12 


0.40 


11, 


147 


4.96 


12. 


11 


0,37 


13. 


205 


6.92 


14, 


105 


3.55 


15. 


92 


3.10 


16, 


26 


0.88 


17, 


105 


3.54 


18, 


146 


4.92 


19. 


222 


7.48 


20. 


53 

2965 


1 ,79 

loOo 


21, 



Subject Qf Request 
Adult Education 

Counseling and Student Personnel Services 
Disadvantaged 
Early Childhood Education 
Educational Administration 
Educational Media and Technology 
Exceptional Children 
Higher Education 
9, Junior Colleges 

Library and Information Services 
Linguistics 
Reading 

Rural Education & Small Schools 
Science and Mathematics Education 
Teacher Education 



Vocational and Technical Education 
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Additionally, cross classifications were made by the seven Colorado 
school districts in the NCBOCES and the various states participating in 
the Center’s activities. Tables reflecting these data are shown in 
Appendix C, 

In summary, those requesting information from the Retrieval Center 
tended to be active educational practitioners, were more frequently 
from the states of Colorado, South Dakota, and Washington, were highly 
favorable toward the service of the Center, and tended to request 
topics related to Educational Administration, Instructional Techniques, 
and Curriculum Development, 



III. Reaction of 1970 Users 



Whereas it was possible to obtain a preliminary indication of user 
reaction to the Center through the standard evaluation form sent to users 
with requested information, it was considered desirable to study user 
reaction in greater depth for the contracted evaluation, A comprehensive 
user questionnaire was constructed to assess in-depth reaction for 
distribution to a sample of those requesting information from the Center 
during 1970. The objectives on which the content of the questionnaire 
were based included purposes for use of the material requested, source 
of information about the Center, reaction to the materials provided by 
the Center, information about educational practices related to the 
material requested, and demographic data of the users. The questionnaire 
which was developed to assess user reaction is shown in Appendix D. 

Since the questionnaire was to be mailed to the user sample, an effort 
was made to present the items In as efficient a format as possible. 

Review of the user information from the Center's files reveals that 
1,088 separate individuals had requested information from the Center 
during 1970, The total population of 1970 users was divided according to 
geographic location so that a stratified sample could be drawn for the 
evaluation. Distribution of the population according to strata and the 
sample which was drawn from it are shown in Table 11. Here it can be seen 
that a 20% sample stratified according to geographic area was drawn for 
the evaluation. The questionnaires were mailed to the sample and were 

ERiC 
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Table 11 

User Questionnaire - Sample Distribution 
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Response 

Boulder Valley Public Schools RE-2J 
Westminster, Adams 50 

Thornton-Northglenn, Eastlake, Adams 12 
Loveland (Big Thompson), Larimer R-2J 
Fort Collins, Poudre, Larimer R-l 
Longmont, St, Vrain Valley, Boulder RE-1J 
Estes Park, Larimer R-3 
Colorado, not in NBOCES 
Nevada 

South Dakota 
Washington 
Oregon 
Utah 
Wyomi ng 

All other States 
Total 



76.1% of questionnaires mailed were returned. 



No. 

1970 


of 

Users 


Sample 


No. 

Returned 


f 


% 




f 


% 


207 


19.1 


40 


34 


20.5 


4 


.4 


1 


1 


.6 


42 


3.9 


9 


7 


4.2 


13 


1.2 


2 


2 


1.2 


20 


1.8 


4 


2 


1 .2 


20 


1,8 


4 


3 


1.8 


10 


.9 


2 


1 


.6 


183 


16.8 


34 


23 


13.9 


23 


2.1 


4 


3 


1 .8 


138 


12,7 


28 


18 


10.8 


16b 


15.3 


37 


31 


.18.8 


44 


4.0 


9 


9 


5.4 


144 


13.2 


27 


19 


11.4 


26 


2.4 


7 


4 


2.4 


_ 48 


4.4 


10 


9 


5,4 


1088 


100.0 


218 


166* 


100.0 
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followed by postal card reminders to return completed questionnaires four 
weeks following the sample mailing. As a result of this l , ^roach, 76.1% 
of the questionnaires were returned. This percentage of returns was 
considered highly satisfactory for purposes of the evaluation. (The 24.9% 
of non-returns included some faulty addresses and some who felt unqualified 
to respond to the questionnaire since they had used the Center only once.) 

In evaluating the validity of the sample distribution, it was noted 
that some distortion may have occurred in that the questionnaire may have 
gone to a contact person rather than to the actual user. Such situations 
were exceptional, however, and it is felt that this circumstance did not 



affect results appreciably. 

Responses to the completed questionnaires were coded and key punched 
directly from the completed questionnaires. Responses to open ended 
questions were recorded verbatim for separate study and analysis. Analyses 
of questionnaire responses were made according to total sample, geographic 
area, occupation, and years of professional experience. It was felt that 
these analyses would provide maximum insights as to reactions of users 
generally as well as specifically. 

In Table 12 are shown the occupations of individuals using the Center 



in 1970. It should be noted that entries in this table as well as those 
which follow represent specific individuals rather than requests (in con 




trast to the analyses in the previous chapter). Thus, the characteristics 
reflected in the 1970 user sample can be considered relatively accurate. 
Inspection of Table 12 reveals that specialists (change agents) constituted 
the largest group of users followed by elementary and secondary school 
teachers and administrators. In comparison to the request analysis of the 
last chapter, the 1970 user data suggest that about the same proportion of 
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Table 12 

Responses to User Questionnaire 
Number of Questionnaires Returned by Occupation 



No. 


Returned 

%_ 


Occupation 


10 


6.0 


1 . 


Student 


14 


8.4 


2. 


Teacher* Elementary 


24 


14.4 


3. 


Teacher, Secondary 


12 


7.2 


4. 


Teacher, college or university 


9 


5.4 


5. 


Librarian 


56 


33.5 


6. 


Specialist, coordinator, director, 
consultant, research specialist 


5 


3.0 


7. 


Pupil Personnel, counselor, psyehol 
school nurse 


19 


11.4 


8. 


Administrator - Principal 


6 


3.6 


9. 


Administrator - Superintendent 


0 


0 


10. 


Layman 


10 


6.0 


11. 


Other 


2 


1.2 


12. 


Teacher, special education 


167 


100.1 




Total 
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teachers requested information in 1970 as in previous years. This situation 
is probably associated with the need for more dissemination efforts by 
Center personnel to encourage teachers to participate in Center utilization. 

In Table 13 are shown the 1970 users by years of professional 
experience. A curvilinear relationship between proportion of users and 
length of experience is apparent from inspection of this table. Individuals 
with more than 15 years experience tended to use the Center less fre- 
quently than those with fewer years experience. It is probable that use 
of the Center as an innovative approach to education is tied to recency 
of trainings level of responsibility and interest in and willingness to 
modify current practice. 

Purposes for which the services of the Center were used are shown in 
Table 14. Inspection of this table indicates that research was the major 
purpose. In preparation of the Questionnaire, "research" was not defined 
per se, and it is difficult to know how the respondents interpreted the 
term. Assuming that their interpretation of research was broad, curriculum 
revision, general knowledge about a subject area, and methods of classroom 
instruction were the most popular utilization. 

Source of information about the Center is shown in Table 15, It is 
apparent that direct contact with professionals through presentations by 
resource center personnel and through work association is a major factor 
in dissemination of information about the Center, Newspapers, bulletins, 
and newsletters played a relatively small role as sources of information. 

The effectiveness of information provided by the Center is shown in 
Table 16. Here it can be seen that more than 80% of the users responded 
that the material was moderately useful or more useful. 
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Table 13 

Responses to User Questionnaire 



Number of Questionnaires Returned by Years of Professional Experience 



No. 

f 


Returned 

% 


Years Professional Experience 


25 


15,0 


Less than 5 years 


31 


18,6 


5-9 years 


43 


25.7 


10-14 yeurs 


35 


21,0 


15-19 years 


19 


11.4 


20-24 years 


14 


8.4 


More than 25 years 


167 


100,1 


Total 



ERIC 
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Table 14 

Rp ponses to User Questionnaire 



Question 1: 


Please check 
services of 


the main purpose(s) for which you used the 
the Resources Center. 


No. of 
Responses 


% 


Response 


53 


16.8 


1. 


General knowledge about subject area 


72 


22.9 


2. 


Research 


14 


4.4 


3. 


Assignments and term papers 


9 


2.9 


4. 


Preparation or updating of 
course bibliographies 


48 


15.2 


5. 


Curriculum revision 


17 


5.4 


6. 


Preparation of a speech or report 


51 


16.2 


7. 


Methods of classroom instruction 


32 


10.2 


8. 


School administration problems 


19 


6.0 


9. 


Other 


315 


100.0 




Total 



O 

ERIC 
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Table 15 

Responses to User Questionnaire 

Question 2: Where or how did you first hear about the services of 

the Resource Center? 



No. of 

Re sponses^ % Response 



58 


29.4 


1 . 


Work associates 


4 


2.0 


2. 


Newspaper 


15 


7.6 


3. 


Bulletins and Newsletters 


22 


11.2 


4. 


Staff meeting 


61 


31.0 


5. 


Presentation by Resource 
Center Personnel 


37 


18.8 


6. 


Other 


197 


100.0 




Total 



ERJC 
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Table 16 

Responses to User Questionnaire 



Question 3: How useful have you found the information provided by 

the Information Retrieval Center? 



No. of 



Responses 


% 


Response 


13 


7.9 


1. Not useful 


17 


10.3 


2. 


43 


26.1 


3. Moderately Useful 


41 


24.8 


4. 


51 


30.9 


5. Very useful 


165 


100.0 




Mean Response: 


3.606 




Standard Deviation: 


1.239 





0 




48 



40 



Pertinence of the material is reflected in Table 17. Again, more 
than 80% indicated that the computer profiles were pertinent and 
more than 70% indicated that tne manual search materials were useful. 

These two percentages would be even higher if the computation of favorable 
responses were based on reactions with "not applicable responses 
el iml nated. 

In Table 18, responses to the availability of information received 
are shown. Almost 90% of the respondents indicated that the material 
received from the Center would not have been available to some degree. 

There was an apparent overlap of only 10% between available material 
and that forwarded by the Center. 

To assess the availability of materials useful in analyzing information 
from the Center, respondents were asked to specify the kinds of materials 
and equipment most lacking in their work situation. It can be seen from 
Table 19, In which responses to this question are shown, that the microfiche 
reader, reader-printer, and duplicator were most frequently lacking. 

It is interesting to note that almost 20% of the users indicated that they 
lack adequate library facilities close to their working situation. 

Problems encountered by the respondents were shown in Table 2C. 
Inspection of this table reveals that 17% encountered no problems, about 
14% found information not pertinent to their request, and 13% indicated 
that their request took too long to process. Profile too general and 
microfiche reader not generally available were indicated as problems by 
l^and 10% respectively. In general, the data in Table 20 showed a 
surprisingly low incidence of problems encountered by the users . 

In Table 21, suggestions for improving the C-nter's services are 
reflected Inspection of this table reveals that expansion of the data 

ERIC 
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Table 17 

Responses to User Questionnaire 

Question 4: Have the abstracts in the profiles (computer) and the 

manual search materials sent you usually been pertinent 
to your request? 



Computer Ab streets 

No. of 



Responses 


% 


Response 


120 


83.3 


Yes 


15 


10.4 


No 


9 


6.3 


Not Applicable 


144 


100.0 


Total 





Manual Search Materials 


No. Of 






Responses 


% 


Response 


87 


71.3 


Yes 


7 


5.7 


No 


28 


23.0 


Not Applicable 


122 


100.0 


Total 



O 

ERIC 
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Table 18 

Responses to User Questionnaire 



Question 5: 


Which one of the following statements best describes 
the information you have received from the Center? 


No. of 


Responses 


% 


Response 


8 


b.O 


All the information I received would 
not have otherwise been available. 


83 


51.9 


Most of the information I received would 
not have otherwise been available. 


52 


32,5 


Some of the information I received would 
not have otherwise been available. 


17 


10,6 


Essentially all of the information I 
received would have otherwise been available 


160 


100.0 


Total 



0 




51 



43 



Table 19 

Responses to User Questionnaire 

Question 6: Which of the following materials or equipment are most 

lacking in your work situation? 



No, Of 
Responses 


% 


Response 




55 


21.3 


Microfiche 


reader 


53 


20,5 


Microfiche 


reader-pri nter 


59 


22,9 


Microfiche 


dupli cator 


41 


15.9 


Government 


reports 


50 


19.4 


Adequate library facilities 
close by 


258 


100.0 


Total 






52 
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Table 20 

Responses to User Questionnaire 

Question 7: Which, if any, of the following problems have you encountered 

in using the services of the Resources Center? 



No, of 

Responses % Response 



30 

13 



35 



24 

17 

26 

33 

17 



43 

11 

249 



12,0 

5,2 

14.1 

9.6 

6.8 

10,4 

13.3 

6.8 

17.3 
4,4 

99,9 



Profile too general 

Too much information to wade t k ough 

Information not pertinent to request 

Too little information 

Complete documents (hardcopy or 
microfiche) not available 

Microfiche reader not readily available 

Took too long to process request 

Confusion as to what kind of information 
the Center can provide 

No problems were encountered 

Other 

Total 
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Keep expanding and updating 

the data base (n=l 23} ' 53 43,1 36 28.5 13 10.6 Is 11.4 8 6.5 2.098 
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base was considered the most critical suggestion followed by shortening 
of the processing time. Accuracy of computer search and specificity of 
request forms were considered least critical suggestions. It is interesting 
to note in interpreting these results that the users were concerned 
with general concepts rather than mechanics in making cheir suggestions. 

Responses to six Likert-type items are shown in Table 22. Here it 
can be seen that the users were generally favorable toward the Center 
with very small percentages of responses to be found in the unfavorable 



range of response continuum. 

In categorizing the instances cited where information from the Center 
had contributed to changes in programs and practices, the Human Factors 
Research Laboratory classification of topics was used. From Table 23, 
it can be seen that curriculum development, instructional procedures and 
techniques, and administrative procedure and organization were most 
frequently cited as areas of impact through Center utilization. These 
results parallel those of the analyses of type of requests, an observation 
which supports the validity of both sets of data. 

Overall, analyses of the user reaction during 1970 reflected highly 
favorable response to the Center. When it is recalled that the questionnaires 
were returned to an outside agency, were anonymous, and provided opportunity 




for negative reactions to be recorded, it can be concluded that response 
to the Center's operation during 1970 was highly favorable. 

Additional analyses were made of the 1970 user reactions according 
to geographical area, occupation, and years of professional experience as 
cross classification enables. These analyses will provide the reader 
with very meaningful, specific interpretations of reaction and can be 
found in Appendix E. Review of the responses by geographic area reveals 
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positive score and 30 being most negative score. 
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Table 23 

Responses to User Questionnaire 



Question 9: Can you cite any instances where information from the Resources 

Center has contributed to changes in programs and practices? 
(Responses categorized according to purpose for which information 
was used.) 



Purpose Category 
Curriculum Development 

Pupil Characteristics, behavior, arid guidance 

Instructional Procedure and Techniques 

Administrative Procedure and Organization 

Teacher Education, pre and in-service 

Subject matter topic 

Special Education 

Instructional Resources, Research 

Other 

Total programs named 



f 




21 

5 

9 

7 

3 

6 

4 
2 

_5 

62 



33,9 

8.1 

14,5 

11.3 

4.8 

9.7 

6.4 

3.2 

8.1 

100.0 




f)7 
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major differences between users in Colorado and chose outside Colorado, 
with Colorado users being more favorable. For example, Colorado users 
responding no problems encountered were 25% versus 12% outside Colorado, 
and the mean attitude scale score for Colorado was 10.29 versus 11.16 
outside Colorado. Analyses of the responses by occupation categories 
indicated that non-admi nistrators used material received for classroom 
instruction improvement in contrast to administrators who used the Center 
for solving school administration problems and curriculum revision. 

Non- administrators tended to be less favorable toward the Center than 
administrators. Interestingly, administrators with more years of professional 
experience who used the Center tended to be slightly more favorable than 
those with less years of professional experience even though they used 
the Center's services less frequently. Apparently, the highly experienced 
educators who do use the Center are those who have a high appreciation of 
the innovation that the Center's concept represents. Verbatim responses 
to the open section of the various questions are shown in Appendix F. 

Perusal of these responses indicates a close parallel to the results 
obtained in the objective portions of the questionnaire. 
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NCBOCES District Interviews 



Administrators in the seven school districts comprising the Northern 
Colorado Educational Board of Cooperative Services were interviewed in 
November or December, 1970, and again in April or May, 1971. Fifteen 
administrators were interviewed in the Fall and eight in the Spring. Inter- 
view schedules for both sets of interviews are shown on the following pages. 

The purpose of these interviews was to obtain information about the 
reaction of users in each district to the service, to find out how the 
user system is organized in the districts, and determine what problems were 
being encountered with the service. An attempt was also made to determine 
what effect the Retrieval Center service is having on educational practice 
in the districts. It was very difficult for administrators to pin-point 
the impact of information received from the Center, however, and only a 
sketchy and subjective estimate of impact was obtained from the interviews. 

The following paragraphs include separate discussions of the Fall and 
Spring interview responses and a summary of all responses. 

Fall Interviews 

Organization of the User System in the Districts 

Organization of the user system varies in each of the seven districts. 

In most districts there is a central office or contact person who takes 
requests from users. This may be the instructional media center (2 districts) 
the elementary or secondary directors (2 districts), the assistant super- 
intendent (1 district), or the principal (1 district). This contact person 
then mails or phones the requests to the Center. In some cases. Retrieval 
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Fall Interview Schedule 



ame 


Ti tl e 


Date of Interview 


Interviewer(s) 


laude Stansberry 


Superintendent, Thompson R-2J 
(Loveland) 


17 Nov, 2:00 p.m. 
(pilot) 


Schwartz and 
James 


. L. Schmelzer 


Asst Superintendent, Poudre 
R-l (Ft Col lins) 


1 8 Nov, 1 :00 p.m. 
(pilot) 


II 


Dbert Turner 


Asst Superi ntendent, Thompson 
R-2J (Loveland) 


23 Nov, 3:30 p.m. 


Cl 


John Stephens 


Superintendent, St Vrain 
(Longmont) 


9 Dec, 9:30 a.m. 


M 


ick Pope 


Asst Suoeri ntendent of 
Instruction, St Vrain 


9 Dec, 10:30 a.m. 


ii 


iirdon Rude! 


Principal, Jr/St High School 
Estes Park 


11 Dec, 9:00 a.m. 


it 


ither Patterson 


Superintendent, Estes Park 


11 Dec, 10;00 a.m. 


H 


I. K. Boltz 


Superintendent, Poudre R-l 


1 1 Dec , 1:30 p.m. 


II 


>b1e Freden 


Consultant, Educational Media 
Boulder Valley 


15 Dec, 10:45 a.m. 


Schwartz 


• . B . Rya n 
also Mr. Gulette 
ind administrators) 


Superintendent, Boulder Valley 


15 Dec, 2:40 p.m. 


Schwartz and 
James 
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imc 


Title 




Date of 


Intervi ew 


Intervi ewer(s) 


Bostrom 


Asst Superi ntendent, Adams 


50 


17 Dec, 


8:0(J a.m. 


Schwartz and 




(Westmi ns ter) 








James 


lice Spangler 


Library Consultant, Adams 


50 


17 Dec, 


8:30 a.m. 


It 


irtin Schmidt 


Title III Director., Adams 


12 


17 Dec, 


10:00a.m. 


10 




(Eastlake) 










ini el B. Stukey 


Superintendent, Adams 12 




17 Dec, 


11:00 a.m. 


li 
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Interviewer: J. James 



Name 



Title 



Date of Interview 



P. L. Schmelzer 


Assistant Superintendent 
Poudre R-l School District 


Apri 1 


o 

C\J 


1971 


A. Spangler 


Library Consultant 
Adams 50 School District 


Apri 1 


22, 


1971 


G. Rude I 


Principal, Jr/Sr High School 
Estes Park 


...... 




I ^ . 


N. Freden 


Educational Media Consultant 
Boulder Valley Public Schools 


Apri 1 


26, 


1971 


A. Reuter 


Assistant Superintendent 
Adams 12 School District 


Apri 1 


27, 


1971 


R„ Turner 


Assistant Superintendent for 
Secondary Instruction 
Thompson R-2J School District 
(Loveland) 


Apri 1 


29, 


1971 


C. Bergman and 
I. Peterson 


Secondary Director and 
Elementary Director 
St. Vrain RE-10 School District 
(Longmont) 


May 3 


, 1971 
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Center staff call the district on a regular basis to collect requests. 
Returning requests are either channeled back through the contact person 
or directly back to the user. Reasons given for channeling requests 
through a central office included enabling the district to collect 
evaluative data about requests and preventing duplicate requests. 

In two districts, users are encouraged to call the Retrieval Center 
directly, or they may go through an intermediary at their own school such 
as the librarian or principal. 

from the foregoing, three different methods of routing requests used 
in the seven districts can be charted. (Steps in parentheses indicate 
alternate routings) 




2 . 




Some districts may use more than one system. System 1 is the most frequently 
used method of routing requests in the NCBOCES. 





P rob! ems Encountered with the System 
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Five of the 15 administrators interviewed indicated that no major 
problems had been encountered up to that time. The most frequently cited 
problem was keeping users aware of the availability of the Center. This 
was thought to be a continuous problem that should be accomplished 
through both personal presentations to school staff and written literature. 
Several other problems were cited related to user training. Thcise included 
difficulty of users to specifically delineate the request so as to receive 
a pertinent profile, lack of knowledge about or negative attitude toward 
the microfiche system, and little knowledge about following up requests 
to obtain hard copy or microfiche. 

Some complaints about the turn-around time were noted by the educators 
interviewed. However, some users were suprised by the speed of the service. 
Reactions to the turn-around time depend upon the expectation of the users. 



Suggestions for Improving the Center 

In response to the question, "Do you think the operation of the 
Retrieval Center can be improved? If so, how?" the administrators responded 
with several suggestions, which are described oelow: 

1. The data base should be continuously expanded and updated. Infor- 
mation about local resources would be very valuable. 

2. A Watts telephone line from the distric to the Center would 
enable reques s to be made more efficiently and economically. 

3. Several educators made suggestions regarding training of users. 
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They felt that users should be encouraged to use the follow-up phase of 
the service. A tape or film to be used by the districts at staff meetings 
was seen as having the best potential for training users by many. Another 
administrator suggested a 1-page bulletin be prepared telling users what 
the Center can do for them and how to contact the Center. 

4. One administrator suggested that the profiles be reviewed by 

the Retrieval Center staff who would made evaluative comment- for the user 
concerning the adequacy of the search and perhaps making suggestions as 
to other avenues of approach not available through the Center that the 
user could himself pursue. He felt that the personal experience and 
expertise of the staff was an asset that should be exploited along with 
the information available from the Data Base itself. 

5, Attempts should be made to shorten the turn-around time as much 

as possible. 



Availability of Microfiche equipment 

Two of the seven districts indicated that no microfiche readers 
were available in their district. These districts were, however, 
able to borrow the portable readers from the Retrieval Center. The 
other districts had one or more regular or portable readers available 
for use in their districts. Some districts had one central microfiche 
printer (hardcopy from microfiche). All districts felt that their 
facilities for use of the microfiche system were inadequate. 




Impact of Information from the Retrieval Center 

When asked if they could identify programs that had been started 
or changed as a result of the use of the Information Retrieval Center, many 

QK 
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educators said this was difficult to assess. Some felt that the use of 
the Center had not been long enough for any impact to be felt. Others 
were able to identify programs where information had been requested and 
received. A few projects were identified where information from the Center 
had been used in finding solutions to the problem. Most of these projects 
involved curriculum planning. 

Response of Users to the Services 

All the administrators interviewed felt that the overall reaction 
of users with whom they had contact was very favorable. Many expressed 
the wish that their district make more extensive use of the services 
available. They mentioned that the Retrieval Center staff had been most 
cooperative and helpful. One administrator noted an overall attitude 
change on the part of many of his staff — he felt that the service had 
enriched the outlook of the staff and increased their ability to promote 
change. 

Overall, the educators interviewed were very pleased with the service 
provided by the Retrieval Center and enthusiastic about its potential for 
promoting educational change. Some felt that their districts had not had 
long enough use of the Center to adequately describe its effect. Some 
problems were encountered with the microfiche system because of lack of 
readers and reluctance on the part of the users to use the microfiche. 
Educators also saw the need for continuing dissemination of information 
lbout use of the Center to users and made suggestions as to how the 
Retrieval Center could aid the districts in accomplishing this goal. 

A few problems were encountered in not properly wording requests resulting 
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in ina-urate profiles being returned. This was seen as a problem in user 
e 'ucation and the request taking process. 
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in inacurate profiles being returned. This was seen as a problem in user 
education and the request taking process. 
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Spring Interviews 

O rganization of the User System in the Districts 

Little change in the system of routing requests was noted in the 
districts since last Fall. Two districts were planning to make minor 
changes in their system. One change involved making the school librarians 
the primary user contacts with the district media center forwarding and 
keeping evaluative records on requests. Previously in this district, 
requests went from the user tc the media center. The other district 
planned to changed the contact office from the assistant superintendent's 
office to the district media center. 

There were two levels of district user contacts in the systems employed 
by the seven districts; 1) an administrative contact for the entire district 
such as the elementary or secondary director, assistant superintendent, 
or media center; and 2) a school contact such as the librarian or principal. 
Some districts channeled requests through both contacts and some through 
only one. 

Five of the seven districts routed requests through a central district 
administrative contact. Two reasons for this centralization were frequently 
given: 1) So that evaluative information regarding number of requests and 

type of requests could be collected. In districts where returning requests 
were also routed through the central office, it was felt that the district 
should evaluate the turn-around time and quality of profiles. One admini- 
strator felt that purchase of district professional materials would be 
facilitated by examination of the type of requests sent to the Center. 

2) Many districts mentioned the desire to prevent duplicate requests 
being sent to the Center from their district. This would avoid placing 
an unnecessary work load on the Retrieval Center staff. 
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In two districts, users contacted the Center directly or through the 
school contact. In one of these districts the administrator interviewed felt 
that a central coordinating office would be a good idea, but stated the dis- 
trict lacked the staff to provide such coordination. In the other district, 
it was felt that the advantages of evaluation gained by routing requests through 
a central office were offset by the delay and possible loss of meaning. 

One administrator observed that there were two peak periods for use of 
the Center. One was from October to January when new programs and methods 
were being tried out, and the other was during the summer when curriculum 
and other committees were at work. 

Suggestions of the administrators concerning the ideal method of routing 
requests are related in the section under "Suggestions for Developing a 
Model for Organization of the User System." 

Reaction of Users to the Services Provided by the Resource Center 

Administrators in the sever, districts indicated that the overall reaction 
of users to the service was very favorable. Some users were quite enthused 
about its possibilities. Some others had made complaints about the service, 
and three of the administrators felt that these arose because the ur* 
not fully understand the nature of the Retrieval Center Service. 

In most districts, not enough use had been made of the newly introduced 
products (CAT, CAP and PET) to provide evaluative information. However, two 
administrators personally felt that these products would be an improvement 
in the service. They felt the catalogs particularly would stimulate requests 
from teachers. One administrator felt that information about the new products 
had not been well received in that district because it was too complicated and 
suggested that a more simple explanation of the products would be useful. 
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P r r> blems Encounterec with the Systen 

As a result of interviews with administrators in the. -even distri-ts, 
three main problem areas were identified. It should be noted that all of 
the administrators were verv pleased with the service and cooperation 
provided by the Retrieval Center staff. Three administrators indicated 
that no important problems had developed so far. The problems that 
were identified were difficulties within the districts in administering the 
service to the users. The areas identified by the administrators as needing 
improvement were as follows: 

1) Dissemination of Information About the Service. 

Three administrators indicated their major problen was how to inform 
potential users about the service. This response was noted both from 
districts who had had extensive use of and districts who were comparatively 
new to the service. The administrators felt dissemination of information 
about the service should be a continuing process or users forget about its 
possibilities. The administrators did not feel that users' merely being 
aware of such a service was adequate, but felt that formation about the 
service should be related to specific needs of the users; and furthermore, 
users should be made aware of what the data base contains, what are its 
limitations and what kind of information is obtained. Further discussion 
of this problem and suggested solutions is contained in the section "How 
Potential Users are Informed of the Availability of the Center." 

2. Difficulty in Properly Delineating Requests. 

The second problem area identified by three administrators was the 
difficulty of many users to properly identify their requests. Many times 
users would fail to give a specific enough description and receive far too 
broad a profile. Other times they would give an ambiguous description and 
receive a profile that was not pertinent. An example . 'en by one admini- 
strator was a request for information on "ungraded" schools. This could 
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mean no achievement grades are given, such as A's, B's, C's, 0's, and l 1 s, 
or it could mean a continuous progress school where there are no grads 
levels, such as 1st grade, 2nd grade, etc. 

The administrators could not give specific suggestions for training 
users so they would be able to place appropriate requests. One suggested 
that it would be helpful if the BOCES request forms were designed in such 
a way as to force the users to be specific. For example, he suggested 
there could be a place to describe for what purpose the user planned to 
utilize the information. Another administrator indicated that he encourages 
district users to follow-up their request through regular district 
channels with a phone call direct to the Center. He felt that the Retrieval 
Center staff could best quiz the users and determine the best descriptors. 

3) Microfiche System: 

The Microfiche system was cited as a major problem within their district 
by four administrators. In these districts, the lack of microfiche readers 
discourages many users from placing follow-up requests. One administrator 
felt the problem was two-fold: 1) Lack of readers conveniently available 

to users, and 2) negative attitude on the part of users toward the microfiche 
system. Other administrators also indicated that users would much prefer 
to read hard-copy than microfiche because they felt the microfiche system 
was a lot of trouble and hard on the eyes. 

Another administrator suggested that BOCES place more emphasis on the 
follow-up process as being the end product rather than the print-out, which 
he felt was only a tool in the information retrieval process. Several 
administrators mentioned that some users expect the print-out itself to 
have the answers to their problems. 
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When asked where the primary responsibility for improvements related 
to the three problem areas rested— with the district or the Retrieval Center— 
the administrators were not sure, but most felt that it was a joint 
problem. They indicated the Retrieval Center staff could help by providing 
suggestions and materials with which the districts can better administer 
the system. 

One administrator had a suggestion concerning the introduction of 
new services such as CAT, CAP and PET. New services, he felt, should be 
fully operational when introduced or a date should be set when requests for 
the new services can be handled. In that district, the reputation of the 
Retrieval Center had suffered when some users did not receive good service 
on the new products as they were first being set up. 

Overall Impact of the Center on Educational Practices 

Five of the seven administrators interviewed stated that assessment of 
the impact of information on educational practice was extremely difficult 
to measure; four added that, if it were to have an impact, it would take more 
time than has so far elapsed since use of the Center began, beveral wtre 
able to identify programs for which information had been requested, but they 
were unable to say whether the information had been read and whether it had 
affected any decisions about the programs. 

Two district administrators could identify programs which information 
from the Retrieval Center had helped to develop. In one district, information 
from the Center had been used in curriculum planning. In another district, 
two areas of Impact were noted: 1) curriculum change. One program in par- 

ticular was described where educators observed a problem in the district 
O and used information obtained from the Center on research done in the area 
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to identify possible solutions and plan a program that was approved by the 
Board. 2) Classroom teachers in this district have used the Center for 
background in subject matter and for information on teaching methods. 

One educator felt that the main impact of the Center was in the 
attitude of users. He felt they were more favorably inclined toward educa- 
tional change now that they had a source of information on educational 
research to aid them in planning new programs. 

How Potential Users are Informed of the Availability of the Center 

In all districts, dissemination of information about use of the Center 
is accomplished in two ways: personal presentations by Retrieval Center 

or district personnel at staff meetings and training sessions, and 
distribution of written newsletters and announcements. In most districts, 
administrators, principals, consultants and media personnel receive the 
personal presentation by Retrieval Center staff; teachers have received 
written materials and/or presentations h’- rK'- 4 -- *ct pers^.ntsj. A 11 
administrators interviewed agreed that educators are being continually bom- 
barded with written materials so that they rarely read carefully anything 
that is not related to their specific needs. It was stressed that 
dissemination of information about the Center should be related to needs 
of the users, i.e. "low can the Retrieval Center help solve my problems," 
and should be accomol ished through personal contact. Three educators 
indicated that dissemination should be a continuing process. 

Si ?cific suggestions offered by the educators when asked for tie ideal 
method of dissemination are listed below: 
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1) Training presentations to users should include description of 
what is contained in the data base, frank admission of its limitations, 
suggestions for possible uses of the Center, description of what 
constitutes an appropriate request and how to place requests, and also a 
description of what is obtained in a profile __ abstracts, print-out, 
manual search, etc. Emphasis should be placed on the follow-up process 
as the actual source of information. 

2) If television is used for in-service training, this would be an 
ideal way to provide continuous up-dating of information about the service. 

If TV is not used and enough staff were available, a presentation by 
Retrieval Center staff to each school at a staff meeting in the Fall 
would be valuable, followed by a brief up-date visit in January. Most 
administrators felt that Retrieval Center staff make the most effective 
presentations to users since they are intimately familiar with the service. 

3) 7t ' ' best if potential users can see an actual print-out from 

the Center in order to visualize how they can best use the service themselves. 

4) If BOCES could prepare a film strip or movie explaining their 
services and showing their Center, this could be distributed among 
individual schools for use at in-service training sessions. 

5) Principals and/or librarians in each school should be given 
primary responsibility for providing continuous information about the 
Retrieval Center service. They should be given some training in how 
to take requests. 

Development of a Model for District Organization of the User System 

In addition to the suggestions for an ideal method of informing 
users about the availability of the Center, the administrators were 
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asked about suggestions for developing a model for organization of the user 
system at the district level. Three of the administrators felt that such a 
model was not appropriate as each district needs to establish a system based 
on their size and organization. Two did feel that guidelines were appropriate 
Four of the educators indicated that the idea of a model for organization 
of the user system was valid. Three of these suggested the system they had 
in their own districts in operation at that time or with planned modifications 
One educator felt the ideal system could not be used in his district because 
of lack of sufficient staff to administer the system. Suggestions and 
observations of the educators concerning the model are presented in the 
following discussion. 

Routing of Req uests: Five of the six administrators who made suggestions 

^or the model indicated that there should be a central office in the district 
through which requests are routed. What office this would be depends upon 
the organization of each individual district. It could be the assistant 
superintendent, the elementary and secondary directors' offices, or the 
media center. The reason for routing requests through a central office, 
according to the educators interviewed, was to enable the district to 
collect data concerning use of the Center. Such data might include number, 
subject, and date of requests, and occupation of user. If returning 
profiles are also routed through the central office, data concerning turn- 
around time and quality of profile could also be collected. However, some 
administrators felt that returning profiles should go directly to the user 
to avoid any further delays. 

A second reason indicated for routing requests through the central 
office was to avoid duplicate requests to the Retrieval Center. However, 
educators felt that a balance should be established between preventing 
ERIC duplicate requests and insuring that materials are available to educators 
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when needed. It was suggested that this balance could best be established 
by sharing requests within a school building, but allowing duplicate requests 

to the Center from different schools. 

Despite the need for centralization, the educators expressed concern 
for keeping the system close to the users to facilitate taking of requests, 
keeping users informed about the service, and decreasing delays. For this 
reason, several educators suggested the districts establish a network of 
"ontact persons in each school who could take requests and provide infor- 
mation about the Center. Most felt the school librarians would be the bes^ 
user contacts. 

One administrator recommended that requests be routed from the school 
contact (librarian or principal) directly to the Retrieval Center. He felt 
that the advantages of evaluation gained by routing the requests through a 
central office were offset by the delay and possible loss of meaning suffered. 

From suggestions made by the administrators interviewed, two basic 
models for routing of requests within a district can be charted: 

School Central RETRIEVAL Central District 

User contact — > District ^ CENTER ^Office and/or > User 

Office School Contact 

School ^ RETRIEVAL 

User * — "5^ Contact ^ CENTER ~ ^ User 

Priorities; When asked if there was a point at which priorities for 
use of the service should be established, the educators replied in terms of 
one of two categories. Three educators based priorities on the type of 
user, indicating that the school districts of the NCBOCES should continue 
to receive priority over other school districts, other states, and university, 
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state department of education or congressional personnel. Four educators 
indicated that priorities should be established according to the type of 
request; requests related to specific problems should receive priority over 
requests related to desire for general information. 

Unmet Needs : Educators were asked if there were any unmet needs which 

could be explored through the Center. Their suggestions were as follows: 

1) The media exchange already being planned by the BOCES was mentioned 
by several educators. One felt that this could be a part of an even broader 
information network among the seven districts that would include public, 
government, and industrial information sources and other local resources. 

2) Information about innovative programs being implemented in schools 
in Colorado would be a valuable aid for other districts in planning and 
selling new programs to the public. Perhaps a questionnaire could be 
developed to send to school districts asking them to describe innovative 
programs in the* : r districts. This could be abstracted and added to the 
data base. 

3) A 1-page flyer listing pertinent issues on which information is 
available could be prepared for the districts to hand out to teachers to 
stimulate requests. 

4) The Current Index of Journals in Education would be a significant 
addition to the data base. 

5) The Center could provide information on what instructional materials 
for the children are available for a specific need, for instance information 
on what materials are available for elementary school children on drugs and 
their use. 
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Summary 



Considerable variation among the districts with respect to organization 
for use of the Center was noted. The patterns of organization can be 
classified into three categories, however. Exchange of information about 
the effectiveness of various organizational patterns appears appropriate. 

Since intense utilization of the Center by personnel in the districts 
is a major objective of the NCBOCES, emphasis on dissemination of the Center's 
activities to all personnel throughout the districts is appropriate. 

Overall j administrators in the districts were pleased with the 
services provided by t'.e Center and enthusiastic about its potential 
for promoting educational change. Evidence of increased familiarity 
with services of the Center was found between the Fall and Spring 
interviews. Coranunicati on about the Center among personnel in the districts 
showed improvement throughout the year. Continuation of tht? i"elationship 
between the Center and the NCBOCES districts should yield significant 
impact on education practice. 



V. Interviews with State Coordinators 



To assess the functioning of the Retrieval Center system throughout 
the participating states, personal interviews were conducted with 
coordinating personnel in Washington, Oregon, Utah, Nevada, South Dakota, 
Wyoming, and Colorado. In some instances only one or two individuals 
were interviewed, but in other instances as many as nine persons were 
interviewed. Although the interviews were planned to cover specific areas, 
an effort was made to keep them relatively unstructured and to encourage 
as much suggestion as possible. Content areas of the interviews included 
organization of the user system in the state, reaction of users to the 
Retrieval Center, problems encountered with the system, overall impact 
of the Center on educational practices in the state, procedures for 
informing potential users of the availability of the service, and suggestions 
for developing a model for state organization of the user system. Inter- 
views were conducted during March, April and May, 1971. 

Because of the great variation among the states with respect to the 
frequency of use, structure of the system within the state, and length of 
time the state has participated, separate reports were made for each state. 
Oral reports were given to Center personnel following the interviews and 
the interviews have been prepared in narrative style for the present report. 
The interview interpretations constitute the remaining sections of this 
chapter. 
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Evaluation of the Information Retrieval Center, Boulder, Colorado 
Washington State Interviews 



Interviewees: 

Title: 

Date: 



Nancy Motomatsu and Assistant 
Associate Supervisor of Learning 
March 2, 1971 



Organization of the User System in the State 

Information Retrieval Center activities in the State of Washington 
are coordinated by Mrs. Nancy Motomatsu, the Associate Supervisor of 
Learning. The Supervisor of Learning's office, organized under the Office 
of Curriculum and Instruction, is responsible for public school libraries, 
audio visual services, and related programs. They operate the state's main 
curriculum library which is an arm of the state library system. The two, 
however, are budgeted and administrated separately. Housed within the 
curriculum library is one of the state's five ERIC microfiche collections. 

At the outset of Washington's involvement with the Information 
Retrieval Center, an advisory committee was established to contribute 
expertise to the office responsible for ERIC. This board was comprised 
of a representative from the Office of Superintendent of Public Instruction, 
a representative from the Washington State Library, and representatives 
from each of the four institutions of higher education which house ERIC 
collections. 

As presently structured, all user requests are processed through the 
office of the Associate Supervisor of Learning. In most cases these 
requests are made directly; however, in some instances an intermediary is 
involved, e.g., a school district staff member. If the coordinating staff 
has any question regarding the request, the user is contacted directly for 
clarification. Processed requests are returned by the Resource Center to 




Washington State Interviews 



72 



send to the user. The purpose of the repackaging is to allow the 
coordinating staff to collect evaluative information such as number 
of profiles returned, turn-around time, and number of manual searches 
received. These data along with the number and kind of requests 
received are systematically recorded. Repackaging and checking takes 
from less than one day to three days. 

The administrative and processing co'-ts associated with t! a project 
incurred by the State have been abso -d b/ the Office of the Supervisor 
of Learning unde' their existing budge - It was pointed out by those 
interviewed that, if Washington is to continue to use Information 
Retrieval Center services, funds above and beyong the $9,000 contractual 
arrangement with the Resource Center will have to be appropriated to 
meet administrative and processing costs. Currently an estimate of these 
additional costs is being prepared. 

Priorities for usage of the Resource Center services have been 
established. These are in order: 

1. Office of Superintendent of Public Instruction staff 

2. Intermediate district staff 

3. Local educational agencies staff — administrators, 
teachers, and supportive staff 

4. School Board members 



5. 
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Reaction of Users to the Resource Center 

To monitor user reaction to the Resource Center service, copies of 
the completed evaluation questionnaire being returned by users to Boulder 
were made and checked. While citing both positive and . ‘ve comments 
made by users, those interviewed felt that the overall r.actir.j was 
positive. Some of the complaints that were received coul have be^ 
avoided if users had a clearer understanding of the limi tc\ti ,r > of t 2 
ERIC system, it was reported. No urban-rural differences in atisf ction 
were noted; however, it was pointed out that the kinds of ~e =sts emanating 
from the two areas differ.. Requests from the latter have be - much more 
specific, e.g., how to set up a mobile film laboratory. Or the other 
hand, an example of a rural request might be "I would like to find out 
what is going on in the area of team teaching." 

Because the three new products, PET, CAT and CAP had not been used 
in Washington, there was limited basis for evaluation. The Associate 
Supervisor's own reaction was quite favorable, however, primarily beccuse 
of the potential of the new products to reduce turn-around time. 

Problems Encountered with the System 

The major problem cited by those interviewed was turn-around time, 
which in Washington usually runs about 3-4 weeks. 

A second problem area involved coordination of request form and 
profile numbers. It was reported that to facilitate processing and 
checking of requests and returned profiles, a common code number should 
be assigned to all materials regarding a specific order. It was suggested 
thc.t if the original question along with its ERIC transition were 
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returned with the profile, it would be helpful. 

More training is needed in taking requests, it was reported. 

The Washington staff did not feel the summer training program in wh" 
they participated was satisfactory in this area. Furthermore, it was 
suggested that out-of-state staff be separated for training from B0( 
staff because of the irrelevance of much of the BOCES material. 

Some complaints had been received that microfiche copies were not 
clear and there was a shortage of readers in outlying districts. Steps 
were being taken to correct this latter difficulty. 

No major problems were cited regarding the quality or quantity of 

information received. 

Finally, the staff in Washington reported they were still waiting 
for the Current Index of Journals in Education to be put on the computer. 

Overall Impact of the Center on Educational Practices in, the State 

The program coordinator indicated that it was too early to assess the 
impact of the Information Retrieval Center service on educational 
practices within the state. However, people involved in changes in 
curriculum, trimester experiments, school year extension studies, among 
others, have used the system and likely have benefited from it, according 
to Mrs. Motomatsu. It was suggested that a systematic evaluation of the 
impact be undertaken during the next year. This evaluation should include 
contacting specific schools or school districts who have extensively 
used the service to determine what changes, if any, have been made as a 
result of the information received. 
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Procedures for Informing Potential Users of the Availability of the Service 



Information regarding the availability of the Resource Center service 
in Washington was disseminated to potential users through various channels. 
Articles were published in "Your Public Schools," a monthly newspaper in 
the state, and announcements were made in workshops and at professional 
meetings. Furthermore, microfiche equipment salesmen were informed of the 
service so that when making calls on potential customers, the program could 
be described. In Washington, a concentrated program to publicize the 
availability of the Resource Center was not made because the coordinating 
staff felt that they could not handle the large number of requests which 
might result. They pointed out, however, that a systematic dissemination plan 
should be coordinated through the state's 14 intermediate districts, as 
well as through all first class districts, a designation made primarily on 
the basis of size. Talks, film strips, sample packets for district workshops, 
and personal contact were all listed as possible dissemination vehicles. 

Suggestions for Developing a Model for State Organization of the User System 
Two possible state organizations were suggested. The first involved 
coordinating the program through the state library system. Libraries in 
Washington are organized into networks made up of 4 or 5 districts so that 
costs of expensive resource materials can be shared by adjacent districts. 

It was suggested that the Resource Center service be set up as an adjunct to 
these networks with a contact point within each network. The contact 
librarians would take requests and forward the request to Boulder where they 
would be processed and returned directly to the user. It may, however, be 
necessary to channel the profile back through the librarian so that evaluative 



Was’ ton State Inte; ews 

records could be maint^'-ed. The coordinating staff was of the opinion 
nat it was critical to minimize the number of intermediaries involved in 
the system. 

A second possibility would be to establish user contact points in each 
of the 14 intermediate school districts in the state. Washington has 
consolidated all of its school districts into 14 intermediate districts. 
This system would be set up so that the user phones his or her request to 
the intermediate district office. They in turn relay all requests received 
to Boulder once each week. The Resource Center would process each request 
and return the profiles to the district who would in turn send them to 
users . 



Interviewee: Dr. Fran Flerchinger 

Title: Manager of Computer Applications 

Date: March 2, 1971 

In addition to meeting with the Associate Supervisor of Learning and 
her assistant. Dr. Fran Flerchinger, Manager of Computer Applications * was 
interviewed. It was Dr. Flerchinger's feeling that the money necessary to 
continue use of the Boulder Resource Center service could be better spent 
developing their own competencies in this area. He cited two primary 
problems with the system as presently structured: 

1. The process of screening documents often contributes to the 
dissemination of irrelevant or incomplete information. 

2. The long distance between Boulder and Washington state results in 
too long of a turn-around time. 
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He would prefer to receive a list of titles not sorted by a librarian. 
The present system is too dependent upon "buzz" word terminology which 
becomes too easily outdated, he reported. He instead proposed a system 
utilizing automatic indexing, where the machine analyzes articles using 
the author's terminology. 



Evaluation of the Information Retrieval Center, Boulder, Colorado 
Oregon State Interviews 

Interviewee: George Katagiri, Director of Instructional Technology 

Jack Bech, Retrieval Coordinator 
Date: March 4, 1971 

Organization of the User System in the State 

The Boulder Information Retrieval service is offered to educators and 
administrators in Oregon through the Oregon Board of Education's Retrieval - 
Dissemination Center in Salem. The Salem Center is headed by George 
Katagiri, Director of Instructional Technology. Jack Bech is the 
Center's Retrieval Coordinator. 

The Center was established on a pilot basis as part of a federally 
funded effort to "close the communication gap" between educational research 
and practice. More specifically, the project was designed to test the 
effectiveness of a dissemination system based upon computerized storage 
and retrieval of information, where personalization of the communication 
process is stressed. 

The program has been primarily oriented toward two intermediate 
school districts in the state, located in Lane and Umatilla counties. 

Each county or intermediate district has a field agent assigned to it 
whose role is to assist in identifying problems, obtaining proper solutions 
and in setting up innovative programs. 

Upon receiving a reguest for information, the field agent forwards 
the problem to the Salem Center where a decision is mace regarding which 
information source cr sources will be used. The Boulder system is only 
one possibility. Otners include the Oregon Total Information Service (OTIS) 
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the cities. The Retrieval Coordinator felt the new products would prove 
particularly valuable in meeting the needs of these rural users. Little 
other evaluative information was available concerning the new products 
as they had just become available at the time of the interview. 

Problems Encountered with the System 

Turn-around time was cited as the major problem encountered with 
the Resource Center Service. In Oregon, requests have been taking from 
three to four weeks to process. Interestingly, turn-around time for 
the three or four PET packages ordered was about the same. 

The Retrieval Coordinator indicated that slightly less than 10% of 
all profiles had been "off base" with respect to the relevance of the 
information supplies. Lack of microfiche equipment was also listed as 
a problem; one which the state has begun to take steps to resolve. 

Overall Impact of the Resource Center on Educational Practices in the State 

Oregon is essentially just getting underway with their dissemination 
program; and therefore, it is too early to assess the impact of the Resource 
Center on educational practices in the state. Essentially, an evaluation 
of the impact would involve an assessment of how well the field agents have 
performed their jobs of facilitating change based on the information 
supplied by Boulder and others. 
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(who have the complete ERIC system on computer), various specialists at 
the Oregon Board of Education and other consulting agencies and individuals 
throughout the state, the Oregon State Library, as well as special 
materials housed within the Salem Center. 

If a decision is made by the information specialist to use the 
Resource Center in Boulder, the problem is immediately forwarded there. 
Currently, about 60% to 75% of all ERIC needs are sent to the Resource 
Center. Boulder returns the profile to Salem where it is inspected and 
sent to the agent who in turn takes it to the user. 

Requests for information are also accepted from the rest of the state. 
These come directly to Salem, by-passi ng the intermediate field agent. 
Systematic records of all transactions with Boulder have been maintained 
and include the request number, origin of request, date received, brief 
statement of problem, agency or person where the problem was directed, 
kind of information furnished, the clients name, and the date the 
transaction was completed. 

Overall Reaction of Users to the Resource Center 

The overall reaction of users to the Boulder service has been good 
according to those interviewed, particularly with respect to the manual 
searches received. One factor cited as contributing to the acceptance of 
the service was the personalized format of the returned profiles. No 
urhan-rural differences in satisfaction were reported. However, as in 
Washington state, the kinds of requests made by rural teachers and 

administrators were more general in nature than request emanating from 
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Procedures f or Informing Potential Users of the Availability of the Service 

Availability of the Boulder Resource Center service has been publicized 
through various channels as part of the service offered by the Oregon 
Board of Education Retrieval Dissemination Center. In addition to 
announcements made in various education publications, a brochure describing 
the Center was prepared and disseminated. However, major responsibility 
for publicity and user education has rested with the two field agents 
who inform educators and administrators of the service on their visits 
to schools in their d'strict. During the pilot stage of the project, 
which will end December 31, 1971, efforts to publicize the program have 
been and will continued to be directed to Lane and Umatilla counties, it 
was reported. 

Suggestions for Developing a Model for State Organization of the User System 

In designing an ideal dissemination system, those interviewed stated 
they felt that a trained cadre of field agents would be an essential 
component. Intermediate district staff, they suggested, are already "spread 
too thin" in Oregon, as well as in many other states, to assume the 
additional task of coordinating such a program. The key to the success of 
a retrieval dissemination service is personalization of the total communi- 
cation process, they reported. Moreover, it is essential to approach 
information dissemination from the point of view of the user and his needs. 
This, they indicated, could most easily be done by a person working 
directly with the; user, helping his identify his problems, obtain informa- 
tion on which to base a solution, and help him implement a program to 
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solve the problem. The field agent has the additional advantage of being 
in a position to identify common problems within his district so that 
resources can be pooled to solve these mutual problems. 

Ideally, it would be desirable to locate a field agent in each 
county in the state. In more populated counties like Multnomah, two 
or more agents might be required. This would necessitate hiring and 
training about 50 field agents in a state the size of Oregon. 

The actual system of requesting and receiving information would 

remain unchanged from that which is currently used, i.e., user > 

agent * Salem Center } Boulder ^ Salem Center ^ Agent ^ 

user. 



Interviewee: Dorothy Alexander, Librarian 

Northwest Regional Education Laboratory 
Date: March 3, 1971 

Information Retrieval Center activities in the State of Oregon were 
originally coordinated by Mrs. Dorothy Alexander, librarian for the 
Northwest Regional Education Laboratory in Portland. The program was 
then shifted to the State Library and from there to its present location 
at the Oregon Board of Education Retrieval -Disseminati on Center in Salem. 

As a result of Mrs. Alexander's early involvement in the project and 
her extensive use of the system, she was asked to give her impressions 
of the Resource Center service. 
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Originally, Mrs, Alexander had proposed that a cooperative dissemina- 
tion service be organized in Oregon involving the state liorarian and other 
state agencies concerned with research and dissemination. Most recently 
she suggested that the system should be coordinated by the State Board of 
Education Library which is separate from the state library. The advantage 
of locating the service within the school libraries is that the user 
contact person would be a librarian trained in information retrieval 
techniques . 

As a frequent user of the service, Mrs. Alexander cited two major 
problems with the Resource Center. First, turn-around time has been 
too long; and second, some of the information screening done by the Center 
has resulted in the return of inaccurate or irrelevant information. 

To alleviate this latter difficulty, it was suggested that the contact 
librarians be trained in descriptor term usage as well as in techniques 
to help users clearly specify their problem. This approach would 
necessitate direct contact with the user, something Mrs. Alexander felt 
is essential. 

Education of potential users should take place through 1) state 
educational programs, 2) inservice educational programs, 3) local and 
regional educational meetings, it was suggested. 

According to Mrs. Alexander, educators in the state of Oregon 
definitely need a system like that provided by Boulder, but indicated 
that its continuation would be only a small step toward closing the gap 
between research and practices in education. 
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Evaluation of the Information Retrieval Center, Boulder, Colorado 
Utah State Interviews 

Interviewee: Kathy Wallentine 

Title: Media Specialist/Reference Center Manager 

Organization of the User System in the State 

Kathy Wallentine, Media Specialist for the State Department of Education, 
coordinates Information Retrieval Center activities in the state of Utah 
as part of a broader Federally funded Technical Assistance program. 

Essentially, this Technical Assistance Program was designed to provide 
rural Utah educators with direct and personalized access to the results of 
current educational research. At the time of the interview, 25 rural school 
districts were participating. There are 40 districts in the state. 

Participating districts were organized under four regional centers located in 
Price, Cedar City, Heber City, and Richfield, Utah. A resource agent was 
attached to each center, whose job it was to collect and clarify requests for 
information on any school issue. These requests were then forwarded to the 
Reference Center Manager in Salt Lake City who in turn would relay the requests 
to Boulder or other appropriate sources. These other sources included various 
research and development centers. State Department personnel, university 
advisors. Bureau of Educational Research, Exemplary Reading Center, Utah Title 
III project, as well as college, state and State Department of Education libraries. 

The return channel for profiles was the same. Evaluative records of all 
transactions were maintained by the Salt Lake City staff. In addition to 
providing rural educators with resource agents, teams of State Department 
personnel were available to assist in program implementation. Administrative 
and processing costs of using the Information Retrieval Center service have 
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been absorbed by the Technical Assistance Program. 

To date, the program has been directed toward educators and administrators 
in 25 rural school districts. Others eligible to use the Boulder service 
include State Department of Education personnel and participants in a 
curriculum writing project in the state. 

Overall Reaction of Users to the Resource Center 

Reactions of users to the Resource Center service tc c£te have been 
generally satisfactory according to the Reference Center i 'anager, wK' 1 has 
monitored evaluation questionnaires being returned to She aiso 

indicated that manual searches have been received better ~ian computer supplied 
information. The Technical Assistance (T.A ) Coordinate , Dr. Kenneth 
Lindsay, was quoted as saying that the Boulder service to date had not been 
a valuable adjunct to the T.A. program because of the long turn-around time 
involved in requesting information from the Resource Center. He was enthusiastic, 
however, about the potential for the new products PET, CAT, and CAP to reduce 
processing time. Only a few new products had been ordered, so evaluative 
information from users was limited. It was pointed out that the first few 
ordered were taking considerably longer to receive than had been anticipated. 



Problems Encountered with the System 

The major problem reported was turn-around time. In Utah it has been 
taking about three weeks to process a request. Furthermore, no significant 
time reduction occurred with the introduction of new products. 
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A second problem involved the inaccurate, irrelevant, or incomplete 
information that has been sometimes supplied by Boulder. This has been 
particularly the case when requests for individual searches have been made 
and new product packets were returned instead. 

Another difficulty involved in this shift was that the identification 
number assigned to the request was not transferred to the returned packet. 

This failure to maintain a consistent identification system was alsc the 
case for some individualized searches. The problem that results from this 
practice is t at the Reference Center Manager has difficulty in matching 
requests with returned profiles. 

Complaints were also received regarding the unavailability of resource 
materials from Boulder, for example, a description of instructional materials 
that might accompany a given reading program. The Utah coordinating staff 
would also like to see the Current Index of Journals in Education put 
on the computer. 

An additional problem resulted from the rural orientation of the Utah 
program. Most users do not have easy access to microfiche readers. Each 
agent has only one. 

Finally, it was pointed out that some of the complaints lodged against 
the Boulder Retrieval Center could be avoided by a better user understanding 
of what ERIC is and its limitations. 

Ov erall Impact of the Resource Center on Educational Practices in the State 

In general , it was too early to assess the impact of the Boulder service 
on educational practices in Utah. However, at the date of the interview. 



Utah State Interviews 



87 



four or five State Department specialist teams had been sent out to assist 
rural educators. In none of these cases were the programs implemented 
related to information received from Boulder. 

Procedures for Informing Potential M sers of the Avail ability of the Service 

Primarily potential users in Utah were informed of the availability 
of the Boulder service through the four regional agents. Personal contact 
has been stressed on visits to sch; ;,ls. The Reference Center staff in 
Salt Lake City prepared and disser nated a brochure describing the program 
aid announcements were made in venous State Department staff meetings. 

Suggestions for Developing a Model for State Organization of the User System 

Reference Center Manager supported the basic concepts of Havlock's model 
of information dissemination. Personalization of the process, she felt, 
was important. Ideally, each region within Utah would have an agent assigned 
to it who would coordinate retrieval dissemination activities. His primary 
job would be to gather information from schools regarding their needs. He 
would then translate these needs and supply information relevant to the 
translated problem. The critical aspect of this system would be starting 
with the needs of educators and administrators. The actual channels of 
information processing would remain unchanged from the system currently 
being used in Utah . 
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Evaluation of the Information Retrieval Center, Boulder Colorado. 

Nevada Interviews 

Intenewee: Victor Hyden, Jr. 

Title: Director, Nevada rr onal Educational Center, Lovel<:k 5 levada 

O rganization of the User System 

Whereas the state of Nevada did not officially subscribe to ne 
information retrieval service provided by the Resource Center in ' oulder, 
the se vice was extensively used by teachers and administrators within 
the 17 county jurisdiction of the Nevada Regional Educational Center located 
in Lovelock. Victor Hyden, director of the Lovelock Center, coorc mated 
Retrieval Center activities within this region. 

All user requests were sent ortelephoned directly to Mr. Hyden's 
office who in turn would relay them to Boulder. The return channel was 
the same. As of the date of this interview approximately 
requests had been processed by his office. Unique to Nevada was the 
procedure of having users, when finished with the information provided by 
Boulder, return it to the Regional Educational Center. The purpose 
was to make these documents available to a larger number of teachers and 
administrators by maintaining them in the Regional Library. 

Users were not charged for the service. However, the Lovelock 
center was charged by Boulder on a per-profile returned basis. These costs 
were absorbed within the Lovelock Center's Title III budget. 

In addition to the Information Retrieval Center, the Regional Educational 
Center had access to the Lockheed Information Retrieval Service located in 
San Francisco. An informal comparison of the Boulder service with Lockheed 
was made by sending each organization identical requests and then comparing 
the accuracy of returned profiles and the processing time to receive the 



Nevada State Interviews 



89 



requested r~ormation. Howe ar, because of the small number of requests 
sent using this procedure -c jsiors regarding the superiority of one 
system ove-" she other were uO~ warranted. In one out of the three tests 
made the ' ~orma ti on suppliec by Lockheed was more accurate in the 
judgement of the user. In the 'ther instances no difference was noted. No 
advantage to either organization was found in the area of turnaround time. 

The Nevada Regional educational Center in Lovelock, a Title III 
project, will end this year "-nd according to it's director it is unlikely 
that the state of Nevada wi’i enter into an agreement with the Boulder 
Resource Center for continued use of its service. 

Reacti ons of users to the System 

In general, the reactions of users were favorable. Opinions of 
users regarding the service were obtained through personal contact and 
monitoring of the evaluation questionnaires used by Boulder. 

No information was available regarding responses to the new products 
offered by the Resource Center; PET, CAT, and CAP. Mr. Hyden did state, 
however, that he felt the new products would be very useful in the rural 
areas served by his Center. 

It was reported that the evaluation questionnaire used by the 
Retrieval Center to assess user satisfaction needed improvement, 
particularly as it related to the state of Nevada. Question number two 
regarding manual searches was irrelevant because vertually no manual 
searches were provided Nevada users. Question number three regarding 
willingness to pay for the retrieval service on a cost-per-profile basis 
should have been more specific it we* '' ported. It was suggested that a 
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Nevada State Inter ?• . 

listing of posset - its be included. Furthermore, it was felt that 
there should have . a question dealing with turnaround time and one 
designed to ider~1~ rom whom the user had heard about the service. 

P ro b 1 ems E nc ou n te ~ ~th the System 

The major prc: with the Retrieval Center service, as reported by 

Mr. Hyden, was tur . . nd time. It was felt that the slowness of the service 

resulted in a drop ~he number of requests made from within Nevada. One 

request was sent b_ Hyden's office two and one-half months ago and 
the profile had not jean returned at the time of the interview. 

Another major problem was the unavailability of microfiche readers, 

There were virtually .3 readers available outside of Lovelock in the 
17 county area served by the Regional Educational Center. 

There was evidence as reported by Mr. Hyden that in some instances the 
information returned ry Boulder was not directly related to the original 
request. Part of the problem, it was felt, resulted from weaknesses in the 
design of the user request form. The form currently used does not lead 
to a point of fine ™finition it was reported. Finally, it was pointed 
out that for a serves of this kind to become effective in a rural area will 
take considerably more time than would be the case in urban districts. 

Overall Impact of the Center or Educational Practices in the State . 

In general, it was reported that Nevada had not subscribed to the 
Boulder Retrieval .. service long enough to assess its impact on 

educational practice? thin the state. 
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H ow are Potential Users Informed of the Availability of the Center . 

In Nevada potential users were informed of the availability of the 
service primarily through personal contacts with Mr. Hyden. In addition, 
bulletin board announcements regarding the service were prepared and 
disseminated to participating school districts. 

Sugg estions for Developing a Model for State Organizations of the Use r 
System . 

It was suggested that ideally the user system should be organized 
through the State Department of Education and definitely not through a college 
or university. It was felt that a university designed and operated system 
would be non-user oriented. Furthermore, it was stated the system should 
not be coordinated through a library. Unlike the situation in urban 
areas, it was estimated that to establish an efficient ana effective 
information retrieval service in rural school districts would take from three 
to five years. 

A critical component in developing such a system would be to 
orient it to the needs of the user. To accomplish the use of field 
agents was suggested. Moreover, it is critical to minimize the number of 
intermediaries involved in processing requests. 

In Nevada political control rests in Las Vegas. This is also where 
the states larger school systems are located and, as a result, it was 
suggested that innovative educational programs could most easily be 
established and conducted there and not in the rural areas of the state. 
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Evaluation of the Information Retrieval Center, Boulder, Colorado 
South Dakota Interviews 

Interviewees: Mr. Art Shaver, Coordinator of Federal Programs, Sioux 

Falls Independent School District #1. 

Mr. Lloyd Duenwald, Director, Title III Project, former 
Director of Educational Service Center 
Mr. Jim Simpson, Coordinator of Retrieval Center Activities 
Dr. William Quincey, Director of Planning and Evaluation, 
State Department of Public Instruction 
Dr. Barnhart, State Superintendent of Public Instruction 

Organization of the User System 

Requests for information are sent to Mr. Jim Simpson, either by phone 
or letter. He transcribes the requests onto the request forms, using the 
handbook of ERIC descriptors to narrow them down and sends them out. If the 
request is urgent, this is done by telephone; otherwise by mail. The 
request profiles are sent to him via the computer, generally at night. Since 
the telephone rates are less expensive at night, he leaves the computer on 
“unattended mode" to receive output from Boulder. Upon receipt, he edits, 
packages and sends the information to the requestors. When the computer is 
working properly (which has been less than one month since its installation) 
turn-around time is approximately one week. Although the manual search has 
been discontinued by the Retrieval Center, Mr. Simpson has enlisted the aid 
of the state library at Pierre in doing them himself. In addition, he consults 
the Bureau of Labor Statistics, Science and Technology Referral Center, and 
the National Labor Relations Board, among others, depending upon the subject 
of the search. 
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Most requests for follow-up on complete documents are also tunneled 
through Mr. Simpson. He gets in touch with the state library, once again, 
and gives them the reference numbers of the articles to be copied. All 
information, whether it be in microfiche or book form, is sent out as 
hardcopy. Consequently, turn-around time on manual searches and requested 
complete documents has been substantially reduced. In addition, few 
problems seen to have developed with the mi crofi che system since little, if 
any, actual microfiche is sent out to the users. Another interesting 
point is that these services provided by the library are completely free, 
including paper. When sending out the initial profiles, he includes a cover 
page stating that full documents can be obtained free of charge compliments 
of the state library. With the advent of the new pre-processed packets, 

Mr. Simpson feels the demand for reproduced materials will decline. 

The retrieval system is actually administered through the library 
system and State Department of Public Instruction. The State Department pays 
the cost for administering the system and the library provides office quarters, 
hard copy reproduction and manual search services. The state does not pay 
for the services, only the administration of the system. The State Depart- 
ment of Public Instruction has allocated money from Title IV, Section 402, 
Resources and Planning, for this purpose. The costs include Mr. Simpson's 
half-time salary and leasing of the computer terminal. 

The service may be used by anyone, but is free only to personnel in 
elementary or secondary education or connected with the State Department 
of Education. Graduate students are required to pay. 

Reactions of Users to the System 

Not many evaluation forms have been returned to Mr. Simpson since some 
are also sent to Boulder. Of the ones he has received, he stated that 95% 
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answered "yes" in response to the question about paying for the service on 
a cost-per-prof i le basis. 

Problems Encountered with the System 

With regard to problems associated with the service, Mr. Simpson suggested 
that dissemination of information about the services provided by Boulder was 
a major problem. Very few teachers have been made aware of the Retrieval 
System. This is evidenced in the vast majority of requests on educational 
administration. When the terminal was initially established, a State 
Department Bulletin was circulated. It generally got only as far as the 
principals, however. Subsequent newspaper articles have made mention of the 
service. Until recently, virtually no formal effort was made to inform teachers. 
Lately, Mr. Simpson has been sending out request forms with a letter informing 
potential users of the Service. The response, he feels, has been much 
better than when information was sent without request ^orms. The main problem 
in his opinion is that people don't understand the written word, and personal 
presentations should be made. He suggested two ways in which dissemination 
can be improved; 

1) When regional and local Education Association meetings are scheduled, 
plan to make a presentation as an adjunct to the discussion topics. His 
point here is that perople would be more motivated to attend this type of 
meeting than if the sole purpose was to disseminate information about 
information retrieval. This way the people are already assembled. 

2) Use State Department personnel to serve a sub-stations throughout 




the state. They could disseminate information and gather requests at the 
same time. 
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With regard to microfiche problems, as was mentioned earlier, no pro- 
blems have yet developed since all documents are reproduced in hard copy 
form. Mr. Simpson mentioned that the Information Retrieval system simply 
could not operate in South Dakota without the aid of the library in 
providing documents. 

Quality and pertinence of information is dealt with at the request 
stage. Since he knows the people fairly well, Mr. Simpson has developed 
an intuitive sense of what they want. In preparing requests, he occasionally 
interprets their requests when they are either unclear or subject to 
misinterpretation. For example, he mentioned a request on "indian" culture. 
He knew they meant "American Indian" as opposed to "India" so he processed 
the request accordingly. In the same manner, he often edits the information 
sent back. As a result, requestors receive pertinent information without 
having to wade through mounds of material. Of course, he runs the risk of 
over-editing. When a search comes up empty or with irrelevant information, 
he questions the user in order to assign more pertinent descriptor terms 
for a second search. 

Turn-around time has been slow compared with what the requestors had 
been used to. Whereas they were accustomed to waiting about one week, they 
now must wait an average of three to four weeks for their requested infor- 
mation. Our cross-classification shows 19.3 and 25.8 for computer and manual 
processing time, respectively. Jim mentioned, however, that the users don't 
seem to complain too much. 

Overall impact of the Resource Center on Educational Practices in the State 

In citing programs which have been modified, begun or influenced as a 
result of using the Retrieval Center, Mr. Simpson mentioned the following: 




South Dakota State Interviews 



96 



1) Watertown used information in developing a continuous edu- 
cational program from grades 1 through 12. He says they were lost 
until they received the Information. 

2) In Weston South Dakota information was reauested on eval- 
uating a Title III Drug Abuse program. They supposedly incorporated 
this information into their evaluation plans. 

3) Information about evaluation instruments and techniques in 
outdoor education was used in one of thi projects. 

4) At the Douglass school information was used to develop a summer 
in-service training program for kindergarten teachers. 

5) Environmental and outdoor education Information was used in a 
summer teach-in. 

6) Attitudes of school planners am builders have changed with the 
receipt of information about new designs in use elsewhere. 

Summary 

Overall, the system seems to be running fairly smoothly in South 
Dakota. Jim Simpson seems to have been doing a competent job in handling 
requests. Although dissemination has been fairly poor, he expects his 
current campaign will boost the number of requests. Should that even- 
tuality occur, the operation would most certainly have to expand. Given 
his present half-time position, he could not handle the flood of requests 
himself. He has applied for a secretary to help him in processing the 
increasing load of requests. At the present time, there are no back-up 
plans should the library's services suddenly become unavailable. It is 
expected, though, that the new products will lessen the severity of this 
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potential happening in that a substantial decrease in the demand for manual 
searches and follow-up of complete documents can anticipated. One 
alternative would be to move the terminal to Vermillion where all the 
resources are housed. The main disadvantage, though would be the loss 
of Jim Simpson. He has been the mainstay in South Dakota and is a valuable 
asset to their Retrieval System. However, South Dakota State University 
seems to have greater possibilities for expansion. One other disadvantage, 
although a temporary one, would be a transitory loss of some requestors. 

In summing up, the system as presently run seems to be operating 
efficiently. However, if the future is to be considered, serious thought 
ought to be given to expansion and the feasibility of its being accom- 
plished at Madison. 
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Interviewees: Mr. Jim Sheehan, Director of Information Management 

Services, Wyoming Department of Education 
Mr. Mel Gillespie, Federal Projects Coordinator 
Mrs. Carol Stearns, processor of all information requests 
Mr. Paul Sanifer, Assistant Superintendent for Planning 
and Development 

Requests for information in Wyoming are sent to Carol Steans in the 
Information Management unit. She in turn forwards the request to Boulder 
either by mail or through Boulder's weekly telephone call. Processed 
information is returned through the same channels. Completed evaluation 
forms are sent to Cheyenne. 

The Retrieval Service is available to all elementary and secondary 
education people as well as State Department of Education personnel on 
a cost- free basis. It is often used by state legislators, board members, 
administrators and, to a lesser extent, teachers, in pursuing government 
projects. Others in the state, such as graduate students, are charged 
on a cost-per-profi le basis. Hardcopy and microfiche may be obtained 
from Boulder or the State Department of Education in Cheyenne since a 
complete ERIC microfiche set is on file there. They do not have a 
microfiche duplicator, but a large reader-printer is on the premises. 
Numerous portable readers (approximately 200) are scattered throughout 
the 125 school districts in Wyoming. No limits have as yet been placed 
on microfiche check-out time. 
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Everyone at the State Department was quite pleased with the 
Retrieval System and its operation. Users have primarily been administrati 
types and they appear to have been well satisfied. Although no specific 
instances or examples could be given with regard to changes in attitude, 
morale, practices, etc., on the part of the users, it was generally 
agreed that the effect of the requested information was positive. 

Very few operational problems with the Retrieval Service were 
defined. The main problem area, according to Gillespie and Sheehan, is 
boulder's desire to see Wyoming use the service more often. As of 
December of 1970, approximately 50 requests had been processed for Wyoming 
requestors (most of which were in October, November, and December). It 
was Sheehan's impression that the lack of follow-up requests for complete 
documents was partially responsible for Turner's less than enthusiastic 
response to Wyoming's participation. Since the State Department has the 
complete ERIC microfiche set, complete documents can be reproduced more 
quickly and at little or no cost to the user (usually free unless an 
inordinate amount of paper is required). 

Turn-around time has engendered both satisfaction and dissatisfaction, 
depending upon user expectations. Little more can be said about that. 
Generally speaking, descriptors are set outside the limits of the request, 
resulting in more information than necessary. This, however, has not 
incited criticism, as most of the information has been quite pertinent. 
According to them., manual searches are still being conducted for them by 
Boulder, unlike the South Dakota situation. 

No money has been specifically allocated for administering the system 
in Wyoming. Administrative costs are absorbed through the delegation of 
additional responsibilities to State Department personnel. It is this 
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area in which Mr. Gillespie feels changes should be made. He suggested 
the assignment of clerical staff with direct responsibility for handling 
requests. It was the evaluator's impression that the current aquest 
load did not warrant such a change. However, innovation of this kind 
could be meaningful when expansion becomes necessary. 

With regard to dissemination, little information about the Center's 
services has been circulated. One article appeared in The Educato r, a 
State Department of Education publication, occasional articles have 
appeared in newspapers, and frequent users sometimes spread the word. 

A campaign is presently being conducted in which 5,100 Wyoming educator's 
are being informed of the Retrieval Service in a circular whose heading 
states, "ERIC offers free research for Wyoming educators." In this 
publication, the July 1 deadline for free searches is mentioned and 
some sense of urgency is conveyed. The purpose is to rapidly boost the 
number of requests from Wyoming, thereby increasing their credibility as 
a future client. It is anticipated that the request load will rise 
substantially during this six month period. 

Both Gillespie and Sheehan suggested the use of workshops and 
highly localized presentations at district meetings for dissemination. 
Sheehan felt there was too little time to accomplish this before July 1. 
The suggestion was made to have an education consultant in the field, 
one of whose functions would be to hold these workshops. Gillespie 
called for the continuous dissemination of data and information collected 
throughout the state to the education community. The purpose would be 
tc stimulate and improve practice at the local level. 
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With regard to the impact of information obtained from the 3 aider 
serv' ir e 5 +'*'o -, nst?.P s .ere ci cea. 

1) Information or learning handicaps and disorders resulted in 
having a program funded for the Cheyenne local school district. 

2) The University of Wyoming College of Education has requested 
and incorporated to a certain extent information on teaching evaluation 
methods in other states. 

In summing up, they are extremely pleased with the service, feel 
it is well-conceived, and are pushing for greater usage in their state. 
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Interviews With 

Cole ;o State Department of Education Personnel 



To obtain the react: ons of the State Department of Education personnel 
to the Resource Center, interviews were conducted with the following persons: 



Dr. 

Dr. 



Roger Duncan, Supervisor 
Charles Beck, Consultant 




ESEA Title III Office 



Mrs. Elizabeth Gibson, Consultant, Improved Learning Unit 



Mr. James Meeks, Assistant Commissioner 

Mrs. Esther Snyder, Reference and 
Interlibrary Loan Specialist, 

Library Services for State Agencies 

Mrs. Ruth Tromley, Consultant 
Field Programs and Consultant 
Services Unit 

Mrs. Bobbye Young, Director 

Miss Joan Harrigan, Asst. Director 

Mr. Hoover, Consultant, 

Library Development and Program 
Coordination Unit 




Office of 
Library Services 



Mr. Edwin Hildebrand, Director of Project SPRtAD 



State Department personnel have been among the most extensive users 
of the Information Retrieval Service in Boulder. However, a few of those 
interviewed had had only administrative contact with the Center 



Strengths of the Resource Center 

At the outset, it is important to point out that all of those interviewed, 
without exception, were of the opinion that the Retrieval Center in Boulder 

was filling a critical education need. Furthermore, all agreed that the 
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Colorado State Department of Education Interviews 

Center was well organized, well managed, and staffed with competent 

personnel. Other strong points of the Resource Center's operation included 
the continual effort made to upgrade and improve the service and, at the 

same time, maintain "personal" contact with users. The effective blend 
of technology with personalization of the communication process was mentioned 
by many as the major strength of the program. 

Specifically, with respect to the profiles prepared by the Center, ‘ 
general satisfaction was noted. The new products, PET, CAP, and CAT, were 
cited as being examples of the continued improvement of the quality of the 
service offered by Mr. McCleary and his staff. Other major strengths 
reported involved the cooperative attitude of the management and staff 
toward developing joint programs with other state agencies, e.g., with 
the Office of Library Services and Project SPREAD. 

Problems Encountered with the Resource Center 



The problems cited with the Center's service were minimal. A few per- 
sons in the library services area felt that turn-around time was too long. 
However, most stated that this was not a problem. In general, the quality 
of the materials provided was judged good, although a question did arise 
regarding differential comprehensiveness of profiles provided graduate 
students as compared to the State Department personnel , with graduate 
students receiving less comprehensive profiles. It should be noted that 
this question was based on a limited number of comparisons and that many 
factors could have accounted for these differences even when identical 
requests are made at different periods in time or even the same period in 
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When queried about unmet needs, several suggestions were made for the 
future development of the Center: 

It was suggested that it would be desirable to have a list of diagnostic 
materials on the computer classified by skill, as well as entries involving 
prescriptive approaches to deal with diagnosed deficiencies. This would 
allow a teacher not only to identify specific pupil weaknesses, but also 
develop a program to remedy the problem identified using the most up-to- 
date materials and techniques. 

The addition of affective, cognitive, and psychomotor behavioral 
objectives to the data base was also recommended along with a description 
of tests available to measure these objectives. Also, it was suggested 
that a list of all teachers in the state by subject area (a list now 
maintained by the State Department) be added to the existing data base at 
the Resource Center. This, it was reported, would allow wider and easier 
access to this information for purposes of administrative planning and 
program development throughout the state. 

It was pointed out that there is a need for clearinghouses to 
systematically examine contradictory entries within ERIC and discard 
outdated materials. In addition, it was suggested that continued 
development of joint programs with other offices within the state should 
be encouraged. More specifically, a coordinated exchange of information 
with Mrs. Lola Quintan of tne South East Metropolitan BOCES, who provides 
an abstracting service in specified areas of education, should be encouraged. 

Many interviewed indicated that the service should be made available 
to more people including not only educators in Colorado, but in all states 
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as well as graduate students and college and university faculties. Specific 
suggestions regarding expansion in Colorado involved installing terminals 
in each Board of Cooperative Service. To maintain the personalized nature 
of the retrieval and dissemination process, one person suggested placing 
pictures of the Resource Center staff in the BOCES terminal rooms. 

To make the service available to a broader spectrum of users, more 
publicity for the Center is needed, it was reported. In addition, potential 
users should receive training in how to make a request. More specifically, 
they should be instructed in how to clearly delineate their problem. To 
accomplish this, in part, some felt that visits to the Center would be 
helpful. It was further pointed out that to assure the greatest possible 
use of the service by teachers and administrators in the state, free 
telephone service to the Center should be provided. 

The management of the Retrieval Center has initiated plans to use 
their leased computer at night to print cnecks , grade lists, etc., to help 
defray computer costs, and this idea received encouragement from members 
of the Title III Office interviewed. 

People in the various offices interviewed have been encouraging their 
staffs to use the Retrieval Service; however, it was suggested more effort 
should be directed along these lines in the future. Moreover, individuals 
submitting Title III proposals have been encouraged to use the service. 

The suggestion was made that when it becomes economically feasible, 
consideration should be given to subscribing to a currier service to assure 
prompt delivery of materials which are often slow in reaching users when 
the mail is used. Considering the lonq range development of the Center, 
some suggested that the Center might be absorbed by the State Library. 
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However, those who advanced this suggestion were quick to point out that the 
staff, organization, and location of the Center would have to remain un- 
changed if it were to be successfully taken over by the library. Others 
argued that the Center would remain independent. 

In addition to the general satisfaction with the specific educational 
need being met by the Information Retrieval Center, secondary benefits 
have resulted from having the Center located in Colorado. On the national 
scene, the Title III National Advisory Board has recognized the Center as 
have some members of Congress, it was reported. Furthermore, it was 
pointed out that politically, having the Center located in Colorado has 
facilitated additional funding of Project SPREAD, the headquarters of 
which are also in Colorado. 

To summarize, all of those interviewed agreed that the Resource Center 
was filling an important educational need and that the Center was well 
organized, well managed, and was staffed with competent personnel. Few 
problems were noted with the service. However, many suggestions were 
made regarding the future development of the Center. These focused 
primarily on increasing the data base and expanding the service to a broader 
spectrum of users. 

Overall, the comments of those interviewed indicated that the Information 
Retrieval and Dissemination Center in Boulder is an exemplary Title III 
project . 
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Summary Statement 

As can be seen from the foregoing interview reports with state 
coordinating personnel, patterns of organization for use or the Retrieval 
Center are extremely varied. This variation is both a strength and an 
area of concern. The strength of the varied organizational patterns among 
states lies in the accommodation of individual differences in personnel 
and educational organization among the states. In other words , since each 
state is unique, it is appropriate that the dissemination system also be 
unique. However, with each state functioning autonomously in the organization 
of the services of the Center, each state must develop its own system. 

Since some errors are being and have been made, comparison of experiences 
among states, without regard to dictating the appropriateness of a 
particular pattern for any given stat^ a should lend efficiency to the 
operation of the Center. Such an approach would avoid the possibility of 
each state making the same mistakes. It is felt that the Retrieval Center 
can improve the functioning of the utilization of the service by offering 
several alternative plans to any state and permitting the state to select 
the plan most appropriate for their situation. This is a plan which will 
become increasingly important as additional state participate in the 
Retrieval Center concept. 
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Retrieval Center Staff Interviews 



Interviews with key personnel on the Retrieval Center staff were 
conducted by the evaluators in May, 1971. The purpose of these interviews 
was to obtain information about the adequacy of the materials used in 
retrieval and the physical facilities at the Center. Also, it was hoped 
that interviews conducted anonymously by outside interviewers would 
uncover problems encountered by the staff or suggestions they might have 
for improving the Center. 

Interviews were directed mainly at the Information Retrieval staff 
but also included some people from other sections who worked closely 
with the Retrieval section. An organization chart of the NCBOCES showing 
the relationship of the various sections is shown on the following page. 

Described in the following sections are interview responses arranged 
according to major topic areas and a brief summary of all responses. 

Adequacy of Data Base arid Manual Search Materials 

It was indicated by those interviewed that the ERIC system needs con- 
tinual revision so that outdated materials are either updated or discarded. 
However, it was felt that tr„ese revisions should be made by the clearing- 
houses with the consultation of experts it specific subject areas rather 
than by the Resource Center staff. User evaluation of materials obtained 
by the Center, however, might aid the clearinghouses in the updating of 
ERIC entries. An additional weakness of the ERIC system cited was that 
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there are few entries in some subject areas, e.g„, art. 

The most frequently cited shortcoming of ERIC centered around the 
procedure for descriptor term assignment. It was pointed ouc that descriptors 
are not assigned uniformly by the clearinghouses and once assigned, are 
vi i oually impossible to change. Moreover, there is no procedure for 
adding descriptors. 

Specific suggestions for adding to the data base included 1) the 
addition of descriptions of projects conducted within the seven NCBOCES 
school districts and the rest of Colorado, and 2) addition of the 
Current Index of Journals in Education. Steps have been taken to 
accommodate this latter suggestion. 

Some of those interviewed stated that more important than increasing 
tie data base entries, is a need to develop and install additional 
procedures to retrieve what is already in the computer. These procedures 
would include adding scope capability to the computer, and ability to 
program searches by identifiers and by exclusion rather than just 
inclusion, it was reported 

Considering the proximity of the University of Colorado library as 
well as local library facilities, general satisfaction was noted with 
regard to materials available for manual searches. It was reported that 
there is a need to develop a more comprehensive file of community resources, 
such as libraries, local consultants, etc., so that when a question cannot 
be satisfactorily answered with the resources available at the Center,, 
recommendations regarding where the information can be obtained can be maue 
to users. 

Another suggestion offered by those interviewed was, the possibility 

of employing subject area specialists for both manual and computer searches, 
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when the Center has grown to the point where this would be economically 
feasible. 



Finally, it was pointed out that, within the production area, there 
is a shortage of staff which has resulted in not being able to keep up 
with current periodical reviews and new product revisions. 

Quality of Products 

All interviewed felt that the new products were a significant improve- 
ment in the service offerred by the Resource Center. Ideally the packets 
should be updated each month; however, procedures for revision are informal 
and little updating has been accomplished since introduction of the new 
products. Although less critical, there are no regular procedures for 
updating manual search files. 

To improve the quality of products now offered by the Center, many 
felt that greater effort should be made to tap user opinions of Resource 
Center materials. 

Adequacy of Center's Equipment and Physical Plant 

Several of those interviewed indicated that the Center was experiencing 
"growing pains;" and as a result, the physical plant was becoming crowded. 
There currently is a lack of space for storage of periodicals, microfiche, 
anu other materials, it was reported. While this problem was not acute 
at the time of the interview, continued growth of the Center will necessitate 
expansion of the physical plant. 




With regard to computer facilities, it was suggested that disk 
capabilities would speed up processing of requests as would installation 
of a scope system designed to allow the operator to see an abstract 
without having to have it printed. Furthermore, by use of this system, 
it is possible to determine the number of abstracts in the data base 
which would be retrieved by usage of a given descriptor. 

User Rapport 

All of those interviewed agreed that relations with users had been 
excellent. One person noted tnat rapport had been better with local 
users than those out of state. This was attributed to personal contact 
possible with local educators which had not been possible with people 
out of state. All felt that personal contact was essential to the success 
of the program and that future growth of the Center should include 
provisions for maintaining a close relationship with users. 

A few suggested that more effort should be made to utilize the production 
staff in the development of materials and procedures which directly 
affect users. 

User Training 

A user training program should contain at least three features, it 
was reported. First, users should be familiarized with the Center and its 
operation. When feasible, visits to the Center should be encouraged. 

Second, users should be advised of what to expect from the Resource Center. 
Specifically, the ERIC system should be clearly explained, and its 
limitation specified. Third, users should be instructed in how to use 
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the Center, including training in how to clearly specify their requests. 

One of the most difficult problems reported was determining the real needs 
of users. Another suggestion indicated that training in the use of 
microfiche was needed. 

Up to this point, some indicated that user information and training 
had been haphazard and that a more systematic plan of education was needed. 
With respect to contact persons or out-of-state coordinators, many felt 
that they were naive with respect to descriptor term usage and problem 
specification. It was, therefore, recommended that these people be given 
training in these areas. 

Suggestions for Improving the Center's Operation 

Several suggestions were made for improving the Center's service. 

These included: 

1. There needs to be better coordination between the program develop- 
ment section and the Retrieval Center. To accomplish this, it was 
recommended that the previous policy of general meetings of the entire 
BOCES staff be reinstituted. These meetings, it was felt, enabled 
better coordination of activities as well as providing the staff with a 
better overall picture of the operation. 

2. To eliminate misunderstanding between management and the production 
staff, it was recommended that a regularly forum between the tw be estab- 
lished. Specifically, it was suggested that the information specialists 

be called upon more frequently regarding design of request forms, user 
evaluation questionnaires, and user education programs and materials. 
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3. Another suggestion invoived using the Retrieval service to 
obtain information on which to base plans for future growth of the Center. 
While the service had been used at the outset for this purpose, it had not 
been used recently. 

4. There is a need to interact more with users, it was reported; 
and as a result, it was suggested that the staff should get out, meet 
users, and discuss use of the Center's service. It was felt that the main 
vehicles for dissemination of information regarding the Center should be 
personal contact and workshops. A systematic dissemination program was 
stressed. 

5. Many of those interviewed indicated that the user request form 
needed revision. From the information retrieval point of view, it is 
important to have a form that not only facilitates a fine definition 

of the problem at hand, but which also encourages specification of the 
greatest number of parameters of the problem so that if one search 
strategy is not successful, others may be tried. 

6. Most interviewed felt that the Resource Center should be concerned 
with evaluating the impact of the information provided on education 
practices. Significant impact, it was pointed out, would be a strong 
selling point for the Center. 

7. Additional efforts should be made to see that all potential users 
have access to microfiche readers. It was suggested that a separate 
project be funded to purchase and distribute microfiche readers. 

8. To provide for the widest dissemination of information in the 
future gr - ■■ ’th of the retrieval service, it was suggested that the 
establishment of a satellite system may be necessary. This system would 
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receive use*" requests from the surrounding district and relay them to 
Boulder for processing. 

9. The possibility was suggested of establishing a system whereby 

requests, which the Center did not have the resources to provide adequate 
information regardir be channeled to other retrieval centers special 

izing in different subject areas, e.g., medicine. 

10. It was suggested that the user evaluation form should be revised, 
that the production staff should be consulted in this revision, and that 
separate user forms should be developed for manual and computer -earches. 

11. More effort, it was stated, should be spent on the long range 
planning of the development of the Center. Again, the production staff 
felt they should be consulted in the planning effort. 

12. One of the most impressive features of the retrieval process 
was the group meetings held by the information specialists. The purposes 
of these meetings were to coordinate search activities and eliminate 
duplication of effort in the retrieval process. All interviewed agreed 
this procedure should be continued. 

Concl usi ons 

There was some indication that as a result of the rapid growth 
of the Center, some breakdown in communication between management and 
the production staff had occurred. If not corrected, it was felt by 
the evaluators that serious morale problems might develop as a result 
of these communications breakdowns. 

Several suggestions were made for improving the Center's operation, 
many of which appear to be worthy of co.^i deration by management and 

ERIC staff. 
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Overall, it was felt by the evaluators that the Resource Center was 
well organized, well managed, and competently staffed. 
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VII. In-Service Education Institute 



To educate personnel directly involved in the functioning and general 
operation of the Retrieval Center, a two-week institute was held from July 
20-31, 1970. The Institute was designed for state and local educators 
and included experiences designed to familiarize participants with the 
Ce n te r . A final report, on the Institute including an evaluation of the 
Institute activities was prepared by Dr. Joseph Daley, Associate Director 
of the Institute, and submitted to the staff of the Retrieval Center. 

Because of the comprehensiveness and accuracy of this report, it is included 
as the remaining portion of this chapter. It was felt by the evaluation 
Project Director that the report itself is a highly appropriate manner 
in which to describe the special institute. Because of the interpretation 
contained in the report, no additional discussion of it was considered 
necessary. 
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I. Introduction: Perspective of the Institute 

This report pertains to the Institute for Program Planning for 
educators from seven Northern Colorado school districts and State Depart- 
ment of Education personnel from Colorado, Utah , South Dakota, Washington, 
Oregon and Wyoming, conducted by the Human Factors Research Laboratory of 
Colorado State University, under' contract with the Northern Colorado 
Educational Board of Cooperative Services and the Colorado State Department 
of Education. 

The general purpose of the Institute was (!’ to familiarize professional 
educators with techniques of empirical problem solving and handling educational 
data from a central source and (2) to lay the groundwork for a cooperative 
interstate network designed to facilitate the dissemination and utilization 
of educational information. 

Practicing educators have a critical need to apply the vast quantities 
of existing educational literature to their every day problems. A Title III 
Project entitled "A Cooperative Community Educational Resources Center of 
the Boulder Valley Public Schools, Boulder, Colorado", was funded for the 
purpose of developing an automated educational information retrieval system, 
designed to identify, organize, store ar.d supply such information to educators 
on request. 

This Conmunity Resources Center provides a communications link between 
the teachers, librarians and administrators and the new and latest research 
and developments in the field of education. It is able to provide the 
school personnel contacting the center with abstracts and suirenaries of new 
educational data in the form of printouts from the computer. In addition, 
manual search services by reference librarians are also available as a part 
~ of the service. 
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On July 1, 1970, the Resources Center became a unit of the Northern 
Colorado Educational Board of Cooperative Services, a cooperative board 
designed to provide program planning and evaluation services to seven school 
districts in northern Colorado. 

Since the beginning of the Community Resources Center it was obvious 
that a real need existed for an educational training unit connected to the 
center. It was felt that educators did not know how to use the Center 
to their full advantage, nor were they trained in the latest techniques of 
empirical problem solving. This institute was proposed as one means of 
meeting this training need. 

Since other states had expressed an interest in making use of the data 
base and services available through the Resources Center the institute was 
also designed to provide interaction with selected state representatives 
necessary to establishing a cooperative interstate network. 

It was also felt that interaction between state and local educators, 
between those responsible for providing information and those requesting it, 
would enrich the total program. 

II. Operation of the Institute 
A. Objectives: 

In order to accomplish the overall purpose of the institute the 
following objectives were established for the participants: 

1. To increase skill and understanding in handling, using, and managing 
materials provided through an Educational Resources Center. 

2. To gain in knowledge of ERIC and retrieval services in general. 
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3. To increase ability to use and evaluate ERIC and retrieval service 
materials. 

4. To increase skill in using a problem solving approach. 

5. To identify and begin to resolve a problem relevant to their professional 
assignment. 

6. To develop a greater appreciation of the role of Resource and Develop- 
ment Centers in Education. 

B. Participants 

Two basic groups were represented in the institute. One group consisted 
of 13 educators with administrative responsibility from the seven school 
districts to be served by the Northern Colorado Board of Cooperative 
Services. The other group was made uo of 9 educators from the state 
departments of education of Colorado, Wyoming, South Dakota, Utah, Oregon 
and Washington. Thus, the institute participants represented those 
educators needing information for purposes of problem solving and program 
planning and those educators responsible for providing the relevant data. 

Both groups, it seemed, could profit from the institute program as 
planned and from interaction with each other. 

C. Institute Program 

As indicated, the primary purposes of the institute were (1) to 
familiarize professional educators with techniques of empirical problem 
solving and handling educational data from a central source and (2) 
to lay the groundwork for a cooperative interstate network designed to 
facilitate the dissemination and utilization of educational information. 

The successive segments of the institute were developed from this basic 
goal . 
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The overall plan of the institute was to combine the knowledge and 
background experiences of the participating educators with the expertise 
of the Institute's staff and selected consultants considered experts in 
their fields. It was hoped this combination of experiences, ideas, 
training and expertise would provide a wholesome atmosphere for the free 
exchange of both concrete and abstract educational thinking related to 
the educational problems and programs of the participants. The results 
indicate that this hope was well realized. 

More specifically, the program of the institute was divided into 
two one-week segments. The first week was designed to provide maximum 
input with sessions in evaluation, problem solving and ’ iformation 
utilization. The participants were also asked to work through two 
simulated problems; one dealing with Differentiated Staffing and the 
other with Planned Program Budgetin' - Systems. Two groups were assigned 
to each problem. Their solutions were then compared wi each other and 
with a third prepared by the institute staff. Early is e first week 
the enrollee's were also made familiar with the Educat nal Resources 
Center and with the process of requesting information jr retrieval. 

The second week was designed to allow the participants to bring 
their new learning to bear on a problem of importance to them and attempt 
to work it through to solution utilizing the full resources of the Center 
and the institute. Essentially, the mornings were used to discuss areas 
of concern to either the local educators or the state department personnel 
and the afternoons were devoted to work on the individual projects. 

The Schedule of Events for the two weeks showing consultants and topics 
is presented in Table I. 
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D. Evaluation. 

Evaluation of the institute was provided for in two ways. 

First, three five-point rating scales were devised to allow the 
participants to rate (1) the value of the weekly components of the 
institute, (2) the extent to which the stated objectives had been realized 
and (3) their satisfaction with the staff and other general aspects of 
the institute. Second, a panel of four participants, two representing 
state level people and two from the local administrators, was identified 
on the second day of the institute and asked if they would be responsible 
for critically responding to the institute and lead an evaluative 
dialogue on the last day. 

The responses gathered at the end of the first week and related to 
the content of that week are summari zed in Table II. In general, the 
responses were positive, the presenters were well received and the material 
covered was both relevant and useful. The highlight of the week appeared 
to be the day spent at the Boulder Center. 

The ratings of topics covered during the second week are sunmarized 
in Table III. Once again the responses were generally positive but with 
more variability reflected than was true of the first week. One explanation 
for this has to do with the two basic groups comprising the institute. 

During this second week some topics were rather directly related to the 
local school administrators and others to the State Department personnel. 
Thus, one group could see the presentation as very applicable while for 
the other group it would have limited value. Also, it was apparent that 
one presentation was too technical and required ? background that the 
majority of t s participants did not have. 
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INSTITUTE EVALUATION FORM 



Indicate the degree of value to you of each of the following parts of 
the Institute.* 







Very 

Low 


Low 


Moder- 

ate 


High 


Very 

High 


1 . 


Managing Changes 
Lynn Svennlng 


Q 


3 


6 


8 


1 


2. 


Boulder Tour 


0 


0 


1 


12 


6 




a. Model for Problem Solving 
Charles 0. Neldt 


0 


1 


6 


7 


6 




b. Information Retrieval 
Bi ll McCleary 


0 


0 


3 


7 


9 




c. Individual Problem 
Description 


0 


I 


7 


8 


3 


3. 


Quality Control Panel 


0 


0 


8 


7 


4 


4. 


Common Problem Simulation 
Exercise 


0 


0 


1 


9 


4 




a. Objectives (Roger Duncan) 


0 


1 

1 


6 


8 


4 




b. Procedures (Joe Daly) 


0 


0 


6 


10 


3 




c. Evaluation (Doug Sjogren) 


0 


0 


2 


11 


7 




d. Group Reports on Problems 


0 


1 


4 


9 


6 


5. 


Entire first week 


0 


0 


3 


11 


6 



♦This table reflects the number of participants responding at each level 
of the scale. 
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Second Week 



I NSTITUTE EVALUATION FORM 



Indicate the degree of value to you of each of the following parts of the 
insti tute.* 





Very 

Low 


Low 


Moder- 

ate 


High 


Very 

High 


1 „ BOCS Panel 


0 


0 


5 


13 


2 


2. Plan for a State Network 


(Jim Simpson) 


0 


1 


10 


8 


0 


3. a. State Plan for Information 


Retrieval (Bobb;, Young) 


0 


4 


12 


3 


0 


b. Discussion 


0 


3 


11 


5 


1 


4. a, State Assessment and Contract 
Accreditation (Colorado State 


Department Staff) 


I 


2 


6 


6 


2 


5. Views on Educational Change 
(Bob Gilberts) 


C 


0 


1 


4 


14 


6. Systems Approach to Planning 


(Nick Gangwich) 


4 


6 


5 


4 


u 


7. Individual Problem Project 


0 


1 


3 


10 


4 


a. Information Retrieved 


0 


0 


5 


9 


4 


b. Individual Study 


0 


1 


5 


9 


2 


8. Entire 2nd week 


0 


G 


5 


13 


1 


9. Total Institute Program 


0 


o 


3 


16 


1 



♦This table reflects the number of participants responding at each level 
of the scale. 
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from the standpoint of the participants it appears that the stated 
objectives of the institute were achieved. These results are shown in 
Table IV. The enrol lees were also well satisfied with the availability 
of the Institute staff and the quality of help received from them 
(Table V). 

While the physical facilities were also quite satisfactory, it is 
apparent from Table V that some of the participants felt the pre- 
institute information was inadequate. It would seem from the variation 
in response, that communication with soma enrol lees was not as complete 
as with others. 

The evaluation panel conducted a useful and well -organized assessment 
of the institute on the final day. Once again, response from the 
participants was positive, reflecting time well spent. One point dis- 
cussed at length had to do with the heterogeneity of the group. While 
there was some feeling that the institute would have been more successful 
if the state and local groups had not been invited at the same time 9 the 
majority feeling was that interaction contributed greatly to the 
program. As one state participant stated, " — I think this has been 
good for us because a lot of times we just have tunnel vision in our 
own areas and we aren't able to understand some of the problems that they 
(local) have." (Alice Spengler) This was reinforced by one of the local 
people saying, "I think It Is short sighted of the network media people 
if they don't feel that they have some responsibility for transmitting 
the other kinds of information that we got In this workshop to their 
users* — you can have the best network In the world and still fall flat 
on your faces if the people who are your users don't have a resource 
o 
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TABLE V 
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Second Week 



INSTITUTE EVALUATION FORM 

C. Indicate your degree of satisfaction with the following aspects of 
the institute. 



Very Moder- Very 

Low Low ate High High 



1. Adequacy of information about the 
institute prior to your arrival. 

2. Physical facilities 

a. Meeting Rooms 

b. Housing (Campus) 

c. Eating (Campus) 

3. Availability of staff for help 
and consultation. 

4. Quality of help received from 
the staff. 

5. Free Time 

6. Institute Program 



0 

0 

0 



0 

0 

0 



0 

0 

0 

1 

0 



1 

0 

2 



3 

3 

3 



9 

6 

4 



6 

8 

11 



9 

2 

14 

11 

10 

7 



♦This table reflects the number of participants responding at each level 
of the scale. 
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Second Week 



INSTITUTE EVALUATION FQRfff 



A. The main objectives of the Institute are listed below. Using the five 
point scale provided. Indicate the degree of attainment of each objective 
at this point in the workshop.* 



Very Mcder- Very 

Low Low ate High High 

1. To increase skill and under- 
standing In handling, using, and 
managing materials provided through 

and Educational Resources Center. 0 0 0 14 6 



2. To gain in knowledge of ERIC 
and retrival services in general. 

3. To Increase ability to use and 
evaluate ERIC and retrieval service 
materials. 



0 0 4 13 3 

0 0 6 10 4 



4. To Increase skill in using a 

problem solving approach. 007 10 3 

5. To Identify and begin to re- 
solve a problem relevant to your 

professional assignment. 005 10 5 

6. To develop a greater appreciation 

of the role of Resource and Develop- 
ment Centers In Education. 0 0 2 10 7 



*Th1s table reflects the number of participants responding at each level 
of the scale. 
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upon whom they can call in terms of how best to use the available 
information." (P. L. Schmelzer) 

The evaluation panel ended with the following observations: 

"I will just say again that it seems to me from the comments that 
you have made that It has been a very positive experience and beneficial 
in various ways to various people." (Noble F re den) 

"I have something here that reflects my attitude about the workshop.. 

The statement says 'the purpose of communication among scientists is 
simply to prevent duplication of effort and to enable scientists to 
profit from and build upon each others work.' I am more convinced of 
that than ever after this workshop. I think that I can profit from and 
build onto the things I have learned you are doing In your states and 
communities. We'll see how that works when I get back." (Dorothy Alexander) 
The overall conclusion appears to be that although some aspects 
could have been improved, the institute program was well-conducted 
and effective. However, the extent to which the program of the institute 
results In better programs and practices In the states and schocl 
districts represented will provide the final evaluation. 

III. Conclusions: 

It would appear that the general purpose of the institute was 
realized. The participant evaluation Indicates that the main objectives 
were accomplished and that the experience was a beneficial one. 

As Is usually the case In such workshops, there were Important 
outcomes in addition to those stated specifically as objectives. Some 
of these warrant comment. 
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Mention should be made of the task oriented nature of this particular 
group. Free time for independent study can often be a problem. This 
group was highly motivated and worked extremely well in completing 
projects and applying the concepts and skills introduced in the formal 
sessions. This willingness to work undoubtedly contributed greatly to 
the success of the institute. 

The staff of the institute and of the Northern Colorado Educational 
Board of Cooperative Services had the opportunity to become well 
acquainted with those educators with whom they will be working during the 
coming year. This cannot help but facilitate the application of program 
planning and evaluation to the problems currently existing in the school 
districts represented. 

The final point can best be illustrated by quoting from the evaluation 
panel discussion. "I have been impressed from the very first day because 
somebody in this workshop established a climate dialogue and reception 
that was more open than I have ever observed in any other similar 
experience. No one from here turned me off. ..There was a free-for-all 
of people with diverse background that is unique in my er^tnence and I 
wish I had the key to developing that kind of atmosphere in groups that 
I may have. I saw better dialogue here than I have with my own staff 
and I got more communi cation experience with a positive nature than I 
have had in any similar experience." (Edward Ronayne) 
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INSTITUTE STAFF 
July 20 - 31, 1970 



Charles 0. Neidt, Director 
Colorado State University 

Joseph Daly, Co-Director 
Colorado State University 

Douglas D. Sjogren 
Colorado State University 

Walter Turner 

Northern Colorado Educational 
Board of Cooperative Services 

William McCleary 

Northern Colorado Educational 
Board of Cooperative Services 
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INSTITUTE CONSULTANTS 
July 20 - 31 , 1970 



Dr. Roger Duncan 

Colorado State Board of Education 
Denver, Colorado 

Dr. Nick Gangwich 
Administrative Assistant 
Jefferson County Public Schools 
809 Quail Street 
Lakewood, Colorado 

Dr. Robert Gilberts 

Dean of College of Education 

2145 Rocky Lane 

Eugene, Oregon 

Charles W. Hoover 
Health, Education, and Welfare 
Office of Education 
ERIC Staff 

400 Maryland Avenue j 
Washington, D.C. 

Dr. Art Olson 

Colorado State nna>~H ->f Educa; . o. . 
Denver, 7? 1 

Dr. Chris Pipho 

Colorado State Board of Education 
Denver, Colorado 

James Simpson 

Southeast Education* 1 Service 
208 East 13th Street 
Sioux Falls, South Dakota 

Dr. Lynne Svenning 
Formerly Communications Consultant 
vith Operat on PEP 
Sausalico, California 
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INSTITUTE ON PROGRAM PLANNING 
July 20 - 31 , 1970 
LIST OF PARTICIPANTS 



Dorothy Alexander 
Li brari an 

N-N Regional Education Laboratory 
710 S.W. 2nd Street 
Portland, Oregon 97204 

Clarence Bergman 

Director of Secondary Education 

St. Vrain Valley Schools 

395 South Pratt Parkway 

Longmont, Colorado 80501 

Lloyd Duenwald 
Title III 

Sioux Falls, South Dakota 57101 

Noble E. Freden 
Media Consultant 
Boulder Valley Public Schools 
P.0. Box 186 

Boulder, Colorado 80302 
Dale uughes 

State Department of Education 
Pierre, South Dakota 57501 

Wi 11 i am P. Larsen 
Information Systems 
Washington State Library 
Olympia, Washington 98501 

Leslie Mi tchell 
Director of Data Processing 
7931 Yates Street 
Westminster, Colorado 80030 

Nancy Motomatsu 
Associate Supervisor 
Learning Resources Services 
Superintendent of Public Instruction 
Olympia, Washington 98501 



Linda McCrea 

Northern Colorado Educational 
Board of Cooperative Services 
1750 - 30th Street, Suite 48 
Boulder, Colorado 80301 

Annette C. Overly 
Librarian-Media Specialist 
Boulder School District RE- 2 
Centennial Jr. High School 
2205 Norwood 

Boulder, Colorado 80302 

Ivan S. Peterson 
Director, Elementary Education 
St. Vrain Valley Public Schools 
395 South Pratt Parkway 
Longmont, Colorado 80501 

Buford Plemmons 
Director of Information 
Poudre School District R-l 
2407 Laporte Avenue 
Fort Collins, Colorado 80521 

Edward Ronayne 

Director of Special Education Service'. 
Poudre School District R-l 
2407 Laporte Avenue 
Fort Collins, Colorado 80521 

J. Gordon Rudel 
Jr-Sr High School Principal 
Park Jr-Sr High Schoo 1 
Box 1140 

Estes Park, Colorado 8051 7 

Melvin Gillespie 

State Department of Education 

Cheyenne, Wyoming 82001 
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P. L. Schrelzer 

Assistant Superintendent of Instruction 
Poudre School District R-l 
2407 Laporte Avenue 
Fort Collins, Colorado 80521 

Martin Schmidt 
Director - Title III - ESEA 
School District #12, Adams County 
10280 North Huron Street 
Denver, Colorado 8022? 



James Simpson 

Research Specialist 

Southeast Educational Service Center 

308 East 13th Street 

Sioux Falls, South Dakota 

Alice Spengler 
Di rector 

Instructional Materials Center 

D1 ?J r l* t No - 50 • Ma ™ County 
7200 Lowell Blvd. 

Westminster, Colorado 80030 






■el Stukey 

Superintendent 
Schcol District #12, Adams 
10280 North Huron Street 
Denver, Colorado 80221 



County 



Robert W. Turner 

Assistant Superintendent of Instructi 
Thompson School District R2-J 
201 South Lincoln Street 
Loveland, Colorado 80537 



on 



Samuel M. Walhfeldt 
Director of Guidance Services 
Poudre R-l School s 
2407 Laporte Avenue 
Fort Collins, Colorado 80521 

Kathleen Hal lenti ne 

Media Specialist 

Utah State Board of Education 

1400 University Club Building 

136 East SoMth Temple 

Salt Lake City, Utah 84111 
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f UL_DR/i □□ 5T,^TE UNIVERSITY 



i ' Hv> i (.nL .LiNS, croc OP?/inn BC3«3ai 



HUMAN FACTQBS aCSEAHCH LABOBATO&V 

June 29, 1970 



It was a pleasure to learn from Dr. Walter Turner of the Northern 
Colorado Educational Board of Cooperative Services that you have 
expressed interest in attending the Colorado State University 
Institute on Handling Educational Information from July 20 to July 
31, 1970. The purpose of this letter is to describe the Institute 
in general and to request preliminary information about your plans 
to attend. 

The Institute will be held on the campus of Colorado State University. 

Fort Collins, the city in which CSU is located, does not 
*h:. 4 a.- direct airline connections, we will furnish ground transporta- 
tion to and from the airports in either Denver or Cheyenne. There- 
fore, it is necessary for us to know your arrival and departure 
times, if you plan to fly. If you drive, complimentary campus 
parking will be furnished. We hope that participants from out of 
town can arrive and be settled by Sunday evening, July 19. Either 
on-campus residence hall rooms (at $6.50 per single or $4.50 per 
double occupancy) or lodging at motels in the area will be avail- 
able to participants. The nearest motel is approximately eight 
blocks from the Institute meeting rooms; the average distance to 
motels is about: twenty blocks. Whereas the residence halls are 
relatively plain, most conference participants who use them consider 
them adequate. 

The introductory session of the Institute will be held Monday, July 
20, at 8:00 a.m. , in Room 180 of the CSU Student Center. At this 
time initial introductions will be made, details of the workshop 
discussed and arrangements made for such matters as registration. 

The remainder of the two week's activities are outlined on the en- 
closed schedule. The typical Institute day will be 8:00 a.m. to 
about 5:00 p.m. We expect to conclude about noon on Friday, July 31. 

From examination of the schedule you will note that there are both 
formal presentations and workshop activity during the two weeks. To 
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assure that Institute participants have adequate opportunities to 
apply the concepts about which they are studying, two kinds of 
practical workshop activities are planned: (1) a common "problem" 

will be identified, researched , and worked through to solution as an 
example exercise, and (2) each participant is being asked to identify 
in advance a problem of particular interest to him that can be used 
as the basis for individual study* 

Coffee and rolls will be available each morning and a catered lunch 
will be provided for participants each noon at no cost. Each partici- 
pant will be responsible for his own evening meals, but all or part 
of the group may wish to eat together in some of the local "gourmet 
establishments" on an informal basis. Dress for the Institute will 
be casual. Since nights can be cool in Fort Collins, a light wrap 
for evening wear is suggested. 

A travel and lodging questionnaire, to be completed and returned at 
your earliest convenience, is enclosed. Although some of the infor- 
mation may s^am unusually detailed, our previous experience with 
Institutes has shown that the more information we have about our 
participants, the more effective we can be in making this Institute 
a highly satisfying and meaningful experience for you. 



Sincerely , 




Charles 0. Neidt, Director 
Institute on Evaluation Techniques 



CON: jo 
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TRAVEL AND LODGING INFORMATION 

Institute on Evaluation Techniques 
Colorado State University, Fort Collins, Colorado 



Name 

Institution 

Position or Agency 

Business Address 

Telephone 

Education : 

Dates Attended Institution Major Degree 



Travel Arrangements - ARRIVAL 





I will be driving my own automobile. 


My estimated 


time of arrival in 






Fort Collins is 


. 




I will be arriving by air at 




on 






Airline, Flight No. , arriving at 










I will need transportation from 


to 


Fort Collins. 






I will be arriving by rail at 




on 






Railroad* arriving at 




. I will need 






transportation from 


to Fort Collins. 





Travel Arrangements - DEPARTURE 

* will be driving my own automobile. My estimated time of departure from 

Fort Collins is 0 

... r will be leaving by air from_ on Airline 

Flight No, at _ . I will need transportation, 

will be leaving by rail from on Rail- 
road, departing at_ * I will need transportaion . 




Continued to next page. 






I^dftlnfc 

I will be staying in the CSU residence 
for the following nights . 



Sun . 


Mon. 


Tues . 


Wed 


Sun . 


Mon. 


Tues . 


Wed 
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halls Single occupancy { $6 . 50) 

Double occupancy ($4. 50) 



Thurs. Fri. Sat. 



Thurs. Fri. 



I will make my own arrangements for lodging. 

T wish to stay in a motel or hotel and would like the Institute Staff to 

reserve a room in my name in the following price range: $_ to$ 

for the following nights: 



Sun . 


Mon. 


Tues . 


Sun. 


Mon. 


Tues . 



Wed. 


Thurs. 


Fri. 


Sat 


Wed. 


Thurs . 


Fri. 





Return Address: Human Factors Research Laboratory 

N125 Morgan Library 
Colorado State University 
Ft. Collins, Colorado 80521 
Te lephone : 491-5206 
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MEMORANDUM 



July 14, 1970 



TO: Institute on Handling Educational Information Participants 

FROM: C. 0. Neidt, Director 

SUBJECT: Final Institute Arrangements 



Details are now being finalized for the institute. Plans are going 
ahead essentially as outlined in our first letter. 

Participants staying on campus will be in Parmelee Hall. Room fees 
can be paid at the registration desk. 

Drivers will meet those participants coming by plane at Stapleton 
Airport in Denver at the times indicated on the information forms which wero 
returned. Participants should meet their driver in front of the bank 
entrance 0£ the lower lobby of the Airport building. This is near the 
baggage claim area. 

Participants arriving Sunday by private car can go directly to Par- 
melee Hall and check in. (A map of campus is enclosed.) 

All institute activities will be held in the CSU Student Center, 
including the noon lunches: Coffee and rolls will be available in the main 

meeting room (Room 180) at 8:00 a«m. * Monday. Formal institute activities 
will begin at 8:30 a.m. 

If you have any questions, please call #303-491-5206 and ask for 
Jeanne; otherwise, we’ll see some of you Sunday and all of you Monday 
morning. 
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Implementation of the Planned Program Budgeting System will be evidenced by 
attainment of the following objectives. 

1. The program components will have been identified and budgeted. 

2. Progress will have been made toward defining objectives for each 
program component. 

3. Alternative procedures will have been identified for each program 
component. 

4. The staff will understand the purposes and functions of PPBS. 

5. The staff will be supportive of PPBS. 

f. A report of school operations will a prepared for the community using 
the PPBS and will be favorably received. 

7. The accounting system will have beer i. organized to the PPBS format 
with minimal disruption. 

8. The system is demonstrated to be operable within the resource con- 
straints of the district. 
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PROCE DURES FOR THE PPBS PROJECT 



1. Identify the program components. 

2. Assign responsibility to appropriate staff to define objectives, 
and Identify alternative procedures and respective costs. (Staff 
involvement should be as complete as possible and continue through- 
out the year. 



3. Reorganize the accounting system to the PPBS fo**~-'t. through inservice 
training of key personnel and/or employment of n Try additional 
staff. 

4. Initiate an Information program to familiarize stai " anr sch :1 board 
and community with PPBS and Its Implications. 

5. Establish a two-way communication channel between PFBS staff nd the 
school staff, school board and community. 

6. Establish a format for periodic special reports to c^o ie ir, ormatlon 
on the progress of the project to be concluded with a tprebensive 
year- end report. 
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DESIGN FOR EVALUATION OF PPBS PROJECT 



143 



The purpose of this evaluation is to assess the effects of the first 
year’s operation of a PPBS system In a school district. The evaluative infor- 
mation will be obtained primarily for the administrative personnel for their 
decision making. Those audiences that are interested in educational account- 
ability should also be considered as corns umers of the evaluation. The -u- 
ation design is presented in the following outline. 



Intents 



Observations 



Antecedents 



1. There is a general under- 
standing of PPBS by the board 
and school staff. 

2. The school staff is not 
unfavorable to the idea of 
PPBS. 

3. A trained staff is available 
for implementing the PPBS project. 

4. Resources in terms of equip- 
ment and materials are adequate 
for implementing the PPBS system. 



Transacti 

1. The PPBS system is imple- 
mented on schedule as outlined 
for the first year. 

2. The implementation of the 
system is accomplished with minimal 
disruption. 

3. The school staff is kept aware 

of the developments in implementation 
of the PPBS system. 



4. The school staff is able to work 
on necessary aspects of PPBS such as 
forming program budgets, defining 
alternatives, and detailing objectives. 



1. Administer a test to a sample 
or conduct interviews. In-service 
type training may be necessary. 

2. Assess attitudes with a scale 
or by interview. 



3. Observation of credentials 
of staff and consultants. 

4. Observation and inventory. 



1. Log of activities of PPBS 
kept on a weekly basis. 



2. Log and observation of 
critical incidents such as late 
payrolls, computer breakdowns, etc. 

3. Log. Description of communi- 
cation between PPBS staff and 
school staff. Periodic inter- 
views with samples of staff. 

4. Description of provision 
for this activity. Interviews 
to assess staff time taken and 
feelings about these tasks. 
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Intents 



Observations 



Outcomes 



1. Each program will have been 
identified and a budget mode for 
it. 

2. Each program component will 
have accomplished an identification 
of alternative procedures and 

a statement of objectives. 



3. The board and staff will 
exhibit increased understand- 
ings of PPBS. 

4. The board and staff are favor- 
able to continuation of the pro- 
ject. 

5. The system is operable within 
the resource constraints of the 
district. 

6. A report of the activities 
of the school is issued to the 
communi ty which exhibits the 
philosophy of accountability and 
the community reaction to the 
report format is favorable. 



7. The accounting and data 
handling system Is revised 
to the extent Intended for the 
first year. 



1. Examine product. Judgment 
of outside panel of adequacy of 
program identification. 

2. Examine products. Again an 
outside panel may judge the ade- 
quacy of the products. The products 
should indicate progress toward 
rather than attainment of the end 
products . 

3. Repeat procedures under ante- 
cedent number one. 



4. Assess attitudes with scale 
or interview. 



5. Audit of year's costs and 
examination of projected costs. 



6. Examine report. Interview 
sample of community to elicit re- 
actions. Be careful to get re- 
actions to format and completeness 
of report, and recognize that 
there will be variance on the con- 
tent. Thus we would hope that 
there is general agreement that the 
report is complete and meaningful. 
On the other hand, there may not 

be general agreement on the Inter- 
pretation of the content of the 
report. 

7. Observation of progress made 
in this area. 
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OBJECTIV r OR DIFFERENTIATE STAFFING PROJECT 



General 



1. The staffing ..ttern will have been implemented as intended in the 
selected areas. This objective will have been realized if the following 
sub-objectives have been attained. 

a. Qualified staff has been employed as defined by the ...attern. 

b. There is minimal conflict and confusion in effecting the patterr . 

c. Students receive more Individual attention and can pursue 
independent study. 

d. The performance of the students is maintained at a high level 
in terms of achievement, school activity, and socialization. 

e. The students^ feel ings about the project and school In general 
are favorable. 

f. The staff maintains interest and commitment to the Differentiated 
Staffing Pattern. 

g. The community is aware of and acceptant of the project. 

h. There is evidence of increased effectiveness of teaching 
technique in terms of greater variety of experiences provided, 
techniques employed, and materials and equipment used. 



2. The staffing pattern will have been implemented within the resource 
constraints of the district. This will be evidenced by: 

a. Adequacy of the operational budget. 

b. Feasibility of projected budgets. 
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PROCEDURES FOR THE PPBS PROJECT 
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1. Identify the program components. 

2. Assign responsibility to appropriate staff to define objectives, 
and identify alternative procedures and respective costs. (Staff 
involvement should be as complete as possible and continue through- 
out the year. 

3. Reorganize the accounting system to the PPBS format through inservice 
training of key personnel and/or employment of necessary additional 
staff. 

4. Initiate an information program to familiarize staff and school board 
and community with PPBS and its implications. 

5. Establish a two-way communication channel between PPBS staff and the 
school staff, school board and community. 

6. Establish a format for periodic special reports to provide information 
on the progress of the project to be concluded with a comprehensive 
year- end report. 
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I. Who is the evaluation for? 

The principle audience of this evaluation Is the administrative staff 
of the school district and the policy board. The information will be 
used primarily for making decisions about the project in general and 
about placement of staff members. Other audiences that would have an 
interest In the evaluation are the staff, the students, the constituents, 
and other schools. 

II. Evaluation Design 

Intents Observations 



Antecedents 

1. The school board and staff 
understood the concept of diff- 
erentiated staffing. 



2. The school board and staff 
are not opposed to Instituting 
differentiated staffing. 

3. A staffing plan Is developed 
that is consistent with the abil- 
ities of existing and/or available 
staff. 



4. Adequate materials and re- 
sources are available for imple- 
menting the plan. 



1. Different observation procedures 
may be used such as: 

a. a test 

b. an Interview 

c. a questionnairre 

The observation would be obtained 
early enough so that decisions might 
be made regarding the necessity of 
some kind of in-service training. 

2. Observation procedures would 
be similar to those for number one. 



3. 

a. Description of staff 
characteristics. 

b. Availability of needed staff 

c. Submit staffing plan to panel 

of 3 experts for judgment of its 
feasibility. 

These observations should also be 
made early (before the program starts) 
to optimize the probability of work- 
able plan being instituted. 

4. Inventory of equipment, material , 
facilities, and other resources with 
judgments made of their adequacy and 
appropriateness for the plan. 
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Intents 



Observations 



Transactions 



1 . The staff members assiane 
roles that are consistent with 
their job definition. 



2. The classroom management 
is conclusive to learning. 



3. There is evidence of Increased Indi 
vidual Ization of Instruction. 

4, Facilities, material, and equipment 
are used to capacity. 



5. The overall plan is Implemented 
as intended. 



Outcomes 

1. The staff maintains enthusiasm 
and committment to the staffing 
situation. 



2. Students are favorable to the 
procedure and school in general, 

3. The performance of the students is 
maintained at the desired level. 

(This outcome is complex and 

should be broken down into its compo- 
nents for the evaluation. Not only 
would one examine different areas 
of performance, but also charac- 
teristics of students). 



0 




15 " 



1. Gather information on this via 
interview and observation. Sample 
staff members at regular intervals 
through the year. 

2 . 

a. Observation of what occurs 

in the classrooms to identify 
teaching techniques used and 
to assess classroom climate. 

b. Interviews with samples of 
students . 

3. Observation 



4. 

a. Observation and interview 

b. Inventory of materials used 
and observe wear and tear on 
equipment. 

5. Observation and interview. 



1. Attitude scales, observation, 
and Interview during the year. Be 
sensitive to critical incidents as 
they might occur. 

2. Same procedures as for number 
one. 

3. 

a. Performance on tests used in 
school testing program. 

b. Performance on teacher-made 
evaluation instruments. 

c. Performance on instruments built 
especially for the program eval- 
uation. 

d. Judgment of quality of student pro- 
ducts like papers, art work, etc. 

e. Evidence on parti ci pat jn in school 
and community activitl.s from 
interview. 

(Continued) 



Intents 



Observations 
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Outcomes (Cont. ) 



f. Administer Instruments or obtain 
data periodically from different 
samples of students. 

g. Be sensitive to critical inci- 
dents and unanticipated outcomes. 

4. The project will be demon- 
strated to be feasible within 
the resource constraints of the 
district. 



4. Audit of budget and analysis of 
projected costs. 
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INSTITUTE EVALUATION FORM 



A. The main objectives of the institute are listed below. Using the five 

point scale provided, indicate the degree of attainment of each objective 
at this point in the workshop. 



1. To increase skill and under- 
standing in handling, using, and 
managing materials provided through 
an Educa ional Resources Center. 

2. To gain in knowledge of ERIC 
and ret rival services in general. 

3. To increase ability to use and 
evaluate ERIC and retrieval service 
materials # 

4. To increase skill in using a 
problem solving approach. 

•*>. To identify and begin to re- 
solve a problem relevant to your 
professional assignment. 

h . To develop a greater appreciation 
of the role of Resource and Develop- 
ment Centers in Education 



Very Moder- Very 

Low Low ate High High 



1 

1 



1 

1 



1 



1 



2 3 45 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 

2 3 4 5 
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INSTITUTE EVALUATION FORM 

B. Indicate the degree of value to you of each of the following parts of the 
institute. 



1st Week 

1. Managing Changes 

Lynn Svenning 

2. Boulder Tour 

a. Model for Problem Solving 

C. O. Neidt 

b. Information Retrieval 

Bill McCleary 

c. Industrial Problem Description 

3. Quality Control Panel 

4. Common Problem Simulation 
Exercise 



Very Mcder- Very 

Low Low ate High High 



2 

2 



2 

2 

2 



3 

3 



3 

3 

3 



4 

4 



4 

4 

4 



5 

5 



5 

5 



a. 


Objectives (Roger Duncan) 


1 


2 


3 


4 


b. 


Procedures (Joe Daly) 


1 


2 


3 


4 


c. 


Evaluation (Doug Sjogren) 


1 


2 


3 


4 


d. 


Group Reports on Problems 


1 


2 


3 


4 


Entire first week 


1 


2 


3 


4 



5 

5 

5 

5 

5 
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INSTITUTE EVALUATION FORM 



Indicate the degree of value to you of each of the following parts of the 
insti tute. 



2nd Week 

1. BOCS Panel 

2. Plan for a State Network 

(Jim Simpson) 

3. a. State Plan for Information 

Retrieval (Bobby Young) 

b. Discussion 

4. a. State Assessment and Contract 

Accreditation (Colorado State 
Department Staff) 

5. Views on Educational Change 

(Bob Gilberts) 

6. Systems Approach to Planning 

(Nick Gangwich) 

7. Individual Problem Project 

a. Information Retrieved 

b. Individual Study 

8. Entire 2nd Week 

9. Total Institute Program 



Very Moder- Very 

Low Low ate High High 



2 

2 



2 

2 

2 

2 

2 

2 



3 

3 



3 

3 

3 

3 

3 

3 



4 

4 



4 

4 

4 

4 

4 

4 



5 

5 



5 

5 

5 

5 

5 
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INSTITUTE EVALUATION FORM 

C. Indicate your degree of satisfaction with the following aspects of the 
institute. 



1. Adequacy of information about the 
institute prior to your arrival 

2. Physical facilities 

a. Meeting Rooms 

b. Housing (Campus) 

c. Eating (Campus) 

3. Availability of staff for help 
and consultation. 

4. Quality of help received from the 
staff. 

5. Free time 

6. Institute Program 



Very Moder- 

Lipw Low ate High 



4 

4 

4 



4 

4 
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Very 

High 

5 

5 

5 

5 

5 

5 

5 

5 



QUESTIONNAIRE 

(midterm) 
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If you have comments, criticisms, suggestions, etc. on any of the following, 
write them in the provided space. 



Administration of the institute 



Facilities for the institute 



Things I hope are covered next week 



Other 
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QUESTIONNAIRE 
(end of session only) 
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1. What two things were most beneficial for you in the institute? Please 
indicate why these have been beneficial. 



a. 



b . 



2. What twc things were least beneficial for you in the institute? Please 
indicate why these have been least beneficial . 

a . 



b . 



3. We would appreciate any comments or suggestions you have about the 

operation of or your relationship with the Northern Colorado Educational 
Board of Cooperative Services. 
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VIII. Reaction of University Teacher Educators 
to the Retrieval Center 



Realizing that the Retrieval Center would be used to a limited extent 
by teacher education individuals -it the university leve , interviews 
were conducted with two university-level educators. Bo~h of these indivi- 
duals carried major responsibilit for certification of under graduate- 
graduate programs. They are indii duals who are also contact with other 
professional educators throughou the region. Because of the possible 
implications of their remarks f:s future operation of the Center, an 
interpretation of their comments is included. 

Teacher educators interviewed felt that the service provided by the 
Retrieval Center is best suited for practitioners. They felt that the 
service would have only limited value at the graduate level in that the 
depth required for training of teachers on the graduate level probably 
would require extensive study in university libraries and would be primarily 
concerned with research reports rather than general publications. They 
were quick to point out, however, that the service probably would be appro- 
priate for "practitioners to be." The reactions were based upon their 
own personal experience and experiences of their graduate students. 

In reporting experiences which other users had had, they indicated 
that reactions of other teacher educators depended upon the degree of 
discrimination required and the depth involved in the study of a given area- 
little discrimination and minimal depth uses resulting in highly positive 
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reactions and high discrimination and maximum depth study resulting in 
negative reactions. They pointed out that there does exist a hop; among 
teacher educators that Retrieval Canter concepts (ERIC, etc.) can become 
valuable contributions to educational practice. They felt that much 
remains to be done in training practitioners to use information t nters 
as pa -t of their professional education. 

oth individuals felt that the main purpose of the Retrieval Center 
should be to assist practitioners rather than graduate students o ' under 
graduate students. To this end, they emphasized the importance o~ training 
practitioners to use the services while in school, but they felt that the 
services had very limited direct application to graduate study. 

Neither teacher educator interviewed felt that the Retrieval Center 
was having a major impact on educational practices at this time and that 
consideration should be given to a larger unit for handling the service 
than a cooperative board. It was suggested that a regional unit would 
be mere appropriate than the Board unit. 

In discussing impact, the need for quality controls on the information 
going into the data base was stressed. Greater precision in the assigning 
of descriptor terms to the original abstracts was also stressed. It was 
pointed out that practicing classroom teachers were probably not in a good 
position to evaluate the validity of research projects reported to them 
and that such intermediate interpretation of projects as are contained in 
the "new products" line would be appropriate. 

It was felt that need for answers to specific problems was the basis 
for motivating educators to use the Center. It was also pointed out that 
many practicing educators may have the erroneous impression that simple 
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answers to complex problems could be obtained from using the Center. 



Both teacher educators expressed doubts about the amount of Federal 
funds being soent for ret~ne and dissemination. They expressed concern 
aoout the '"'ormation g c*nc ir *o the Center and the general apathy of 
educators regarding use of such systems as ERIC and the Center. Both 
stressed the need for more accurate categorizing of information and for 
comprehensiveness in coverage. 

In sunrma ry, the teache educators interviewed felt that the value 
of the Center rests with 4-t s use by practitioners rather than educational 
theorists or graduate researchers. To this end, they suggested that 
training to use the retrieval concept should be a part of the professional 
preparation of teachers, but chat administration of the Center should rest 
with a regional rather than a board of cooperative services unit. They 
also emphasized the need for accurate descriptorizing and comprehensiveness 
of information put into the data base. 




L reus si on and Recommendations 

This report has be~ :~ganized into autonomous sections so that 
reference could be made i-dly to data from a particular source related 
to a given facet of the ... . -ation of the Retrieval Center. This method 
of organizing the report .e resulted in discussion and recommendations 
located within each cha_ Rather than duplicating those portions of 

each chapter in this ser , the evaluation will be discussed in relation 
to specific questions lilted in the proposal. These questions are 
directed to the general concept and overall operation of the Center rather 
than to specific facets or its operation. They are also useful frames 
of reference for presenting the final recommendations. 

Who Uses the Center Most Frequently? 

Detailed consideration of this question was presented in Chapters 
II and III. A summary ^=sponse to the question is as follows: "Practi- 

tioners (Administrators, specialists, and classroom teachers) who are 
interested in changing tne manner in which they fulfill their professional 
functions." In considering this abbreviated response, emphasis should 
be placed upon practitioners as opposed to theoreticians, innovators versus 
individuals wishing to perpetuate the status quo, and problem solvers 
versus individuals see ^ to apply new ideas in the absense of problems. 

. t are a response to this i estion can be made conclusively, however, the 
assumption that various kinds of professional educators had equivalent 
knowledge of the Center nv : . be met. As the Center operated chrough 1970, 



this assumption could not be made. In other words, it is conceivable that 
the relatively large proportions of administrators and specialists using 
the Center (in relation to the size of their population) in contrast to 
the lesser proportion of classroom teachers using the Center (in relation 
to the size of their population) may be a function of knowledge about the 
Center and convenience in using the Center rather than a difference in 
interest. It will be interesting to follow user characteristics in a 
longitudinal fashion in the years ahead to determine whether the numbers 
of teachers using the Center will increase proportionally as they become 
more familiar with its functions and its availability. It is likely that 
the generalization will hold, however, that those who use the Center are 
practitioners and innovators. 

What Uses Do Educators Make of the Information Services Provided 
by the Center? 

Detailed response to this question was included in Chapters III, IV, 
and V. In general, the data indicated that the information supplied by the 
Center is being applied to decisions about immediate operations of schools. 
The most frequent use of the Center appeared to be in instances where a 
problem exists and a problem solver is seeking possible solutions or in 
instances where a preliminary solution has been identified and background 
information about it is needed. There was relatively little evidence that 
the information supplied by the Center was used for personal development 
of the requestor or the satisfaction of the requestor's curiosity about 
personal questions. 
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What Questions Coming to the Center are Most Prevalent? 

As indicated in Chapter II, administrative procedure and organization, 
instructional procedure and techniques, and curriculum development were 
the subject matter areas most frequently involved in requests. Since the 
Center's inception, greatest proportionate increase has occurred in the 
category administrative procedure and organization. In terms of specific 
"non-admini strati ve" topics, most frequent reference has been made to 
counseling and student personnel services, library and information services, 
science and mathematics, and social science. 

What Ki nd s of Materials Are Needed at the Local Level? 

Generally speaking, users have been well satisfied with the materials 
provided by the Center (except in those instances where microfiche readers 
were unavailable), but they indicated a desire to learn about innovative 
programs in their geographic area; opportunities to inspect teaching 
materials so that specific choices might make adoption decisions; and 
community resources for reference in teaching. Whereas these kinds of 
materials are beyond the scope of any dissemination Center to provide, they 
are suggestions for directions to be explored in the future. Evidence 
obtained in this project suggests that supplying information to schools is 
only the first of several steps necessary to produce educational change. 

The Center may wish to explore participation in additional stages of the 
change process in the future. If so, addition of other services must be 
explored. 
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What Impact Does the Center Have on Local Educational Practice? 

With the design of the present investigation, it was not possible to 
assess the impact of the Center on actual practice accurately. Likewise, 
participation in Center activities over long periods of time will be 
necessary to answer this question fully. On the other hand, evidence was 
obtained that changes associated with Center requests can be identified 
by administrators of participating units. Further, evidence was obtained 
from users themselves that information provided by the Center was applied 
to the solution of problems. These kinds of evidence suggest that the Center 
is having some, although an indeterminate amount, of impact on educational 
practice. Much more elaborate and costly procedures of investigation than 
were possible in the present project will be required to answer this question 
adequately. 

What Factors Contributed Toward the Motivation of Educators to Use the Center? 

Data obtained from all sources consulted in the present study indicate 
that the solution of problems is the primary factor contributing toward 
motivation to use the Center. A secondary factor related to the general 
improvement of educational practice. Both of these factors are, of course, 
predicated on the assumption that prospective users are familiar with the 
existence of the Center and have its services readily available to them. 

Geographically Speaking, How Much Area Can One Service Center Cover? 

Evidence from the present evaluation suggests that one service center 
can cover a geographic area comprised of several states when it is assumed 
that I) prospective users can attend training sessions to familiarize them 
with use of the Center, 2) referral networks are available which make it 

convenient to use the Center, and 3) close liaison exists between field 
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coordinators and Center staff. Obviously there is a limit to the geo- 
graphic area which can be served by a Center, but available data suggest 
that this limit is outside a several-state area. In considering this 
question, it is important to recognize that the states included in the 
present project are "sparsely-populated" or "low density" states. Exten- 
sion of any generalizations drawn from the present study must consider the 
s.ize of the population served as well as geographic area. 

How Should Informational Materials Requested at the Local Level be Stored 
in Terms of Magnetic Tapes, Discs, etc.? 

Very limited evidence suggests that magnetic tapes are satisfactory 
for Centers with limited use, but that discs are more convenient for 
large centers. Additional study of this question is warranted as the 
Center gains additional experience. 

What Kinds of Remote Facilities and Services Are Most Effective 
at the Local Level? 

This question is considerably more complex than initial reference to 
it might indicate. A variety of services at remote locations were involved 
in the present study, but each had been designed with several aspects of 
the local situation in mind. For example, from the standpoint of the requestors 
alone, any system which is efficient and convenient will suffice. From the 
standpoint of monitoring requests and avoiding duplication as well as from the 
standpoint of tabulation, summarizing, and re-packaging services, more facili- 
ties are appropriate. It would appear that, for handling requests generated 
by users, some centralizing of requests prior to transmittal to the Center 
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is desirable. For the dissemination of current awareness materials 
and pre-packaged sets of information, intermedi ate handling of requests 
from the users to the Center should be kept at a minimum. Thus, in 
responding to this question, it is necessary to consider the nature of 
the request itself and the kind of information being disseminated. 

What T raining Programs are Most Effectiv e for Encouragi ng Local Educators 
to Use the Services Provided by the Center? 

As indicated in Chapter VII, the concept of an institute appears to 
be a very effective method for familiarizing users with the Center. A 
complete response to this question will require experimentation with several 
training methods; but at this stage of the Centers operation, concentrated 
personal experiences appear to be appropriate. 

How Should a Center Such as the Educational Resources Center be Staffed 
and Organized? 

Without formally defining the objectives and functions to be fulfilled 
by any given center, it is not possible to answer this question in a valid 
manner. Given the kinds of objectives associated with the Retrieval Center 
at Boulder, however, some suggestions can be made as follows: Components 

of the staff should include general management, technical personnel (program' 
ming specialists), input specialists, output specialists, and training and 
public relations personnel. In the present investigation, individuals 
concerned with production (output) indicated a need for constant communica- 
tion with all other facets of the organization. Because of the role of 
these individuals, it appears that organizational structures of centers 
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should allow greater interaction between them and other center personnel. 
The production individuals appear to be the key factor in the success of 
the Center and their functions should dictate the Center organization. 

In the present investigation, it was apparent that the procedure of 
having output specialists meet to share their suggestions for handling 
each request was a major contributor to the success of the Center. Since 
each output specialist brought to the consideration of a request a unique 
background of experience in education and a unique degree of familiarity 
with the data base, it follows logically that the group decision and 
subsequent plan for handling the request was greatly enhanced by such 
communication. This practice is considered by the evaluators as highly 
desirable and as having many implications for the staffing and organization 
of similar centers. 

It would also appear that 3.s a center becomes large, specialization 
by subject matter area of input arid output specialists is appropriate. 

The field of education is so broad that one individual cannot adequately 
deal with all kinds of topics. The point at which specialization is 
necessary will, of course, depend upon the size of the center and the 
complexity of the population it serves. 
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Recommendations 

Innumerable reconmendati ons have been made throughout the body of 
this report. Most of them relate to specific act, ities of the Center 
rather than to the overall functioning of the Center as a totality. 

The following recommendations are more general than those contained in 
the body of the report and relate to the overall mission of the Center’s 
operation in the future. 

1. Research on Dissemination 

There is a critical need in education at the present time for the 
validation of theoretical models of dissemination on which to base 
retrieval center activities. The gap between educational practice and re- 
search findings is extremely wide. It is the opinion of the present 
evaluators that this gap cannot be closed appreciably until appropriate 
models of dissemination are developed. The evaluators believe that every unit 
concerned with dissemination has an obligation to devote some of its 
activities to the development of appropriate dissemination models. It is 
therefore recommended that some attention in the Retrieval Center at 
Boulder be directed toward research designed to validate dissemination 
models. As the models are validated, change in the operational procedure 
of the Center will be dictated. In the absence of valid models, 
recommended changes are at best sketchy and incomplete. The Retrieval 
Center offers an excellent vehicle for research in the area of dissemination 
and this potential should be exploited fully. 
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2. Adequacy of the Data Base 

At the present time, the data base of the Center consists of infor- 
mation obtained from a variety of sources. As might be expected, informa- 
tion from some sources is excellent and information from other sources 
is questionable. It is therefore recommended that the material coming into 
the Center be evaluated carefully for its validity and that questionable 
materials be excluded from the base. This type of input evaluation will 
require expertise in many subject matter areas as well as a procedure 
for making the evaluations, but such a step is considered highly 
desirable by the evaluators. Within the vicinity of the Center are many 
educational experts whose services for evaluating materials in the data 
base could be utilized. Such a program would contribute substantially 
toward the credibility of information provided by the Center and would, 
in the long run, contribute substantially to the Center's effectiveness 
and impact. 



3, Completeness of the Data Base 

In addition to continuous assessment of the validity of information 
in the data base, there is also a need to assure completeness of information 
in the base. In fact, the long range success of the Center is highly 
dependent on validity and completeness of data. It is recommended that the 
Current Index of Journals in Education be used extensively for the base 
along with descriptions of programs which are innovative in participating 
schools. Continued interaction with the ERIC data base is also recommended. 
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4. Turn-Around Time 

With the introduction of the new products early in 1971, it was 
anticipated by the Center staff that turn-around time would be reduced 
appreciably. It is the opinion of the evaluators that the new p»x)ducts 
will permit much greater efficiency with which information, particularly on 
highly interesting topics, can be disseminated. In the case of exten- 
sive manual searches, it is recommended that the Center identify sources 
of specialization related to particular topics so that outside assistance 
can be obtained for manual searches. It should be noted that the impact 
of the new products was not assessed in the present evaluation. 



5. Organization at the Local Level 

As was indicated in the body of this report, the opportunity for 
a particular school system to develop procedures for relaying requests 
to the Center is a desirable feature. It is recommended that additional 
assistance be given schools in organizing a procedure appropriate for any 
given school based on experience of participating districts. It is 
suggested that four or five approaches be designed and presented to each 
prospective school so that some choice among procedures is allowed but 
that schools would not have to repeat mistakes made in other situations. 
In general, it would appear that the approaches developed in the BOCES 
schools during the year would accommodate most prospective participants. 




6. Continuous Evaluation 

The Center is to be commended for accumulating evaluation data as the 
Center developed. The form currently used, however, is not adequate for 
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all purposes and should be revised. It is recommended that the Center 
staff assemble various evaluation devices (ERIC, HFRL questionnaire, etc.) 
and develop a brief evaluation format which can be used directly with 
requestors. It is also recommended that a similar form be developed for 
occasional use with state coordinators. Since these forms can be made 
highly objective, quantification and plotting of trends is possible. 

Such a practice would yield an indication of reaction to the service at 
two levels over a period of time. 

7. Awareness of the Center 

Shortly after the need for greater information about the Center in 
the field was reported to the Center staff by the present evaluators, 
increased emphasis was placed on public relations. The evaluators 
obtained some evidence that this was a desirable move, and it is 
recommended that the present emphasis on public relations and awareness 
of the Center be continued. Audio tapes, video tapes, slide presentations 
and so forth all should be useful for extending awareness of the Center's 
services to proposed users. It is also suggested that members of the staff 
visit users frequently and that "open house" visitations to the Center 
by users be encouraged. 

8. Descriptor Term Assignment 

Throughout the evaluation, reference was frequently made by users and 
by Center staff to the need for refinement of the assignment of descriptor 
terms to a given project. Some progress in this area was made during the 
evaluation period, and it is recommended that continued attention be given 
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to this area. Modification of the ERIC descriptor catalog system ^long 
with the development c"? a u 1 . que 'Ll .anomy based on the Retrieval Center s 
experience is appropri ate. Additionally, there is need for training of the 
coordinators in the assignment of specific requests to informational signals. 

9. Library Referrals for Manual Searches 

As was noted in Chapter III, reaction of the users to the services 
of the Center, response to the manual searches has been highly positive. 
Manual searches have been extremely time consuming, however, and there- 
fore costly. It is recommended that a network of manual search sources 
be defined so that not all manual searches must be made by the Center staff. 
State departments and district information resources should be helpful in 
this regard. Thus, rather than eliminating manual searches, it is suggested 
that assistance with them be obtained. 



10. Impact of the Center on Education 

In the early stages of the evaluation, it was hoped that an accurate 
assessment could be made of the impact of the Center's services on educa- 
tional practice. This was not possible, however, and it is recommended that 
further exploration of this area be continued with future evaluators. At 
best, specific observers could be p ced in schools to identify changes asso- 
ciated with the Center's services. At least, follow-up study of uses made 
of Center information should be continued. Achievement of the assessment of 
impact requires considerable time, however, and the longitudinal aspect of 
such a study is imperative. Admitedly, the objectives of the Center include 
primary emphasis on dissemination only, but much could be contributed to 
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education in gener? if the impact of the Center's information could be 
demonstrated. In t absence of complete data, it appears that several 
stages or steps ar= necessary between a request for information and 
ultimate change in educational practice. Dissemination of information is 
only one link in the entire chain, but study of the "links" beyond 
dissemination is highly desirable. This recommendation is closely related 
to Recommendation 1. 

11. Local Equipment 

Because of the problems associated with microfiche, it is essential 
that microfiche readers be located a convenient places for users or that 
convenient and inexpensive systems be available for hard copy to be 
distributed. It is recommended that continued efforts be made to make 
hard copy more readily available to users. 

12. Computer Equipment 

It is recommended that continued study be given to the possibility 
of using discs for storage and retrieval rather than tapes. There appears 
to be efficiency in scaning and storing through the use of discs. In-depth 
analysis of the equipment situation will be required before conclusions 
can be reached. 

13. New Products 

It is unfortunate that the present evaluation did not provide concrete 
evidence related to the evaluation of the new products developed by the 
Center. Limi^d evidence suggests that these materials are highly effective, 
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however, and it is recommended that evaluation of them be undertaken 
immediately . 

14. Expectation About Center Services 

On occasion throughout the Evaluation period, some indication was 
encountered that users had a much higher expectation of the Center's 
services than was realistic. It is recommended that in all contacts with 
prospective users, coordinators by cautioned to establish realistic 
expectations about the Center's services so that disappointment will be 
avoided. Greater familiarity with the services and functioning of the 
Center will also contribute toward establishing realistic expectations. 

15. BOCES Assistance 

Two distinct audiences were served by the Center during 1970. One 
audience consisted of individuals indirectly related to the Center 
through participation in the Center's activities by their state, and 
the other audience consisted of individuals served directly by the 
Center through the BOCES districts. The diversity of these two audiences 
suggests that two levels of service be defined: one for those indirectly 

served and the other for those directly served. It is recommended that 
a closer coordination among Program Development, Program Evaluation and 
Information Retrieval within the BOCES schools be developed. For example, 

BOCES participants could be given the opportunity to request consultant 
services regarding a topic at the time that it is submitted as an informa- 
tion retrieval request. This would alert the consultants to potential 
programs and to interests developing in the schools. With this recommendation. 



173 



it should be possible to assess the stages through which an innovative 
idea must pass before it is implemented, and a major contribution to 
educational theory could be made. This recommendation is closely related 
to Recommendation #1 regarding research on dissemination. Seldom is such 
an opportunity to study a phenomenon as readily available as exists in 
the relationship between the BOCES and the Retrieval Center, and it is 
strongly recommended that this potential be exploited. 

16. Information About Local School Programs in the BOCES Districts 

It was frequently noted that practitioners had a desire to learn 
more about innovative programs being implemented in other schools in 
their area. It is recommended that a local system of in-depth information 
about innovative programs be placed in the data base and that this infor- 
mation be dissemination to participants. Once the information is disseminated, 
it can be followed up by various visits and training institutes. Tne use 
of the computer for handling such a phenomenon has yet to be demonstrated 
and such a demonstration could in turn be generalizable to other situations. 

Summary 

In summary, the Retrieval Center in Boulder is considered to be an 
effective unit by the evaluators. Response to the service is favorable and 
some evidence exists that the Center is having an impact on educational 
practice. Recommendations for change in the Center's operation relate 
primarily to refinement and extension of present activities. 
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Appendix A 



Proposal for Evaluation 
of the Retrieval Center 



EVALUATION OF THE COOPERATIVE 
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RESOURCES CEMTER 
Boulder , Colorado 



The primary obj ectives of the Cooperative Community Educational 
Resources Center, Boulder, Colorado, are to l) reduce the delay between 
current educational research and classroom practice, and 2) increase 
professional growth through individual research assistance. These 
primary objectives are accomplished through several specific strategies 
as follows : 

1. A "current awareness service" wherein monthly abstracts of new 

information and resources relating to professional needs of users are 
sent to educators having filed an interest px'ofile of typical areas of 
concern to them; * , 

2. A computer-based retrieval system for providing educational 
information on reques classified according to ERIC abstractors, and 
local description; and 

3. Manual search services to identify educational information, such 
searches being made by reference librarians upon request of educators - 

Educators using the resources center directly are located in eight 
districts in eastern Colorado* Educators using the resources centex 
through extension terminals are located in South Dakota, Utah, Vr/omi ng , 
and the Northwest Regional Educational Laboratory at Portland, Oregon. 
Because of the complexity of the Center, a comprehensive evaluation of 
the project will require an examination of several dimensions as follows 

a) Characteristics of users 

(1) geographic location in relation to the Center 

(2) position within' their school 

(a) .field 

(b) level 

(c) experience 
. (a) education 

(e) previous experience with research 

.(f) demographic characteristics 
(age. sex, tenure, etc.) 

b) Content of information requested 

(1) top?.cal content of material 

(2) source of information (periodical, book, etc.) 
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(3) use to be made of information 
c) Ultimate impact on educational practice 



(1) 


change in behavior or functioning of user 
(morale, knowledge, attitudes, etc*) 


(2) 


change in practices of users (techniques and methods) 


(3) 


Change in content of educational experiences 
(what is taught) 


(4) 


change in behavior of pupils of'users 



d) Efficiency of dissemination process 



(i) 


time required for turn around 


(2) 


stages or delays in obtaining information 


(3) 


cost of providing information 


(4) 


relationships with data sources other than Center 


(5) 


internal organization and functioning of Center 

(a) advisory board 

(b) administrative personnel 

(c) professional personnel 


(6) 


Antecedents 

(a) publicity 

(b) needs 



e) Quality of information provided 



(i) 


adequacy, currency, and completeness of data bank 

% 


(2) 


readability and format of information supplied 


(3) 


validity of data supplied for intended purpose 


(4) 


•absence of problems in obtaining information 


(5) 


professional service of Center personnel 



f) Physical setting and hardware 



(i) 


kinds of main frame hardware best suited for Center 

1 


(2) 

ERIC (3) 


i 

peripheral equipment best suited for Center ! 

. ■ i 

hook-ups for most efficient results 






( 4 ) load on computer at scheduled times 
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g) Software and storage 

(1) suitability of descriptors and information retrieval 

(2) sequence of coding and storage of information 

(3) sources of information 

(a) ERIC 

(b) local information 

h) Training program for users 

(1) type of training 

(2) location 

(3) length 

(4) content 

( 5 ) training staff 



The evaluation effort will be designed to permit both process evalua- 
tion and product evaluation. In general, the central purpose of the 
evaluation effort will be to provide complete or partial answers to the 
following questions through the collection of appropriate evidence: 

1. Who uses the Center most frequently? 

2. What uses do educators make of the information services provided 
by the Center? 

3* What questions coming to the Center are most prevalent? 

4* What kinds of materials are needed at the local level? 

(Government reports? Audio-visual materials? Abstracts?) 

5. What impact does the Center have or. local educational practice? 

(The research will be conducted only within the Northern Colorado 
Experimental and Developmental Center.) 

6. What' factors contribute toward the motivation of educators to use 
the Center? 

7 * Geographically speaking, how much area can one service cente’r cover? 
8. What kinds of materials are needed at the local level? 




How should informational materials requested at the local level 
be stored in terms of magnetic tapes, discs, etc-.? 
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9. 



10 . 
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What kinds of remote facilities and services ore most effective 
at the local level? 

11. What training programs are most effective for encouraging local 
educators to use the services provided by the Center? 

12. How should a Center such as the Hducational Resources Center be 
staffed and organized? 

To obtain data for answering the foregoing questions.) detailed records 
will be maintained about the search requests made of the Center. So that 
personal characteristics data will not be duplicated unnecessarily, each 
user will be requested to complete a personal characteristics questionnaire 
only once. All subsequent requests made by this individual will then be 
coded to his personal characteristics data* 

Content of questions (all questions will be cleared through USOE) u ; 
typo of material needed for application at the local level will be assessed 
through a brief use questionnaire which will* be based on interviews with a 
sample of users. Purpose of the interview/ will be to define categories of 
use and type of materials needed so that an objective checklist can be 
developed for use with all requests. Development of such a checklist will 
make data collection highly efficient and will provide a continuous flow of 
information about requests and materials which can be monitored by the 
Center administration. 

Impact of the Center On educational practice will be assessed in two 
ways. First, a sample of educators using the Center 9 s services will be 
interviewed to determine' changes in educational practice initiated by them 
in relation to the service requested. Second, schools served by the Center 
geographically will be surveyed for new program activities and these will be 
scrutinized for possible relation to the services provided by the Center. 

Depth interview will be conducted with a sample of users to determine 
their motivation for using the Center's services. These interview responses 
will provide insight into the kinds of service desired at the local level 
and the kind of training preferred by local educators. 

Geographic area coverage will be studied by determining the distance 
of requests from the Center itself and from terminals throughout the area. 
Saturation of requests within concentric circles away from the Center will 
be used to describe the nature of the relationship between distance and 
frequency and type of requests. Comparisons between districts with and 
without terminals but equidistant from the main Center will yield suggestions 
for future organization of remote units. 

t 

Process evaluation will involve detailed study of the organization and 
efficiency of the functioning of the Center in relation to administrative- 
practice. For example, turnaround time for handling requests before and 
after specific changes in organization will suggest the relative effici 
of new administrative pre~tices or modified organizational structure 
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Coding Key 

Requests made to NCBOCES 



Information Retrieval 



Center 



Columns Data Code 

1_4 Request Number 

0001 through 29 — 



5-6 



State or District 

01 - Boulder Valley Public Schools, RE-2J 

02 - Westminster Adams 50 

03 - Thorn ton-Northgl enn , Eastlake, Adams 12 

04 - Loveland (Big Thompson) Larimer R-2J 

05 - Ft. Collins, Poudre Larimer R-l 

06 - Longmont, St. Vrain Valley, Boulder RE-1J 

07 - Estes Park, Larimer R-3 

08 - Colorado not in NCB0CES 

09 - Nevada 

10 - South Dakota 

11 - Washington 

12 - Oregon 

13 - Utah 

14 - Wyoming 

15 - All other States 



7_8 Occupation or Title 

01 - Student 

02 - Teacher, Elementary (K-6) 

03 - Teacher, Secondary, Jr. High and High School 

04 - Teacher, College or University 

05 - Librarian . „ . 

06 - Specialists, Coordinators, Directors, Consultants 

07 -• Pupil Personnel - Counselor, Psychologist 

08 - Administrator - Principal 

09 - Administrator - Superintendent 

10 - Layman 

11 - Other 

12 - Teacher, Special Education 



9-10 




Institution 



01 ~ Elementary School 

02 - High School, Jr, Hi. 

03 - College or Univ. 

04 - State Dept of Educ 



05 - Research Organization 

06 - District Admin Unit 

07 - Teacher Training Institution 

08 - ERIC & Other Resource Ctrs 

09 - Other 

10 - Special Education Institution 




181 



Col umns Data Code 

11-12 Blank 



13-14 Date of Request 

01 - Prior to 1958 

02 - January to June 1968 

03 - July to December 1968 

04 - January to June 1969 

05 - July to December 1969 

06 - January to June 1979 

07 - July to December 1970 



15-17 Computer Processing Time in Days 



18-20 Manual Processing Time in Days 



21 



Type of Profile sent 

1 - Cl only 

2 - H only 

3 - M only 

4 - Cl and H 

5 - Cl and M 

6 - H and M 

7 - Cl, H and M (all three) 



22 Question #1 on Evaluation Form 

(No. of Computer abstracts useful) 

1 - 100% (all) 

2 - 75% (most) 

3 - 50% (some) 

4 - 25% (few) 

5 - 0% (none) 



23 Question #2 on Evaluation Form 

(No. of Manual search materials useful) 

same as above for column 22 



24 



Question #3 on Evaluation Form 
(Would you pay for service?) 



ERIC 



1 - yes 

2 - no 



192 



Columns 



Data Code 



25 



Additional Comments on Evaluation Form 



1 - Generally positive 

2 - Positive with reservations 

3 - Negative 

4 - Neutral 



26-27 



HFRL Subject of Request 



01 - Curriculum Development 

02 - Pupil Characteristics, behavior and guidance 

03 - Instructional Procedure and Techniques 

04 - Administrative Procedure and Organization 

05 - Teacher education, pre and in-service 

06 - Subject matter topic 

07 - Special Education 

08 - Instructional Resources, Research 

09 - Other 



01 - Adult Education 

02 - Counseling & Student Personnel Services 

03 - Disadvantaged 

04 - Early Childhood Education 

05 - Educational Administration 

06 - Educational Media and Technology 

07 - Exceptional Children 

08 - Higher Education 

09 - Junior Colleges 

10 - Library and Information Services 

11 - Linguistics 

12 - Reading 

13 - Rural Education & Small Schools 

14 - Science and Mathematics Education 

15 - Teacher Education 

16 - Teaching of English 

17.- Teaching Foreign Languages 

18 - Tests, Measurements and Evaluation 

19 - Vocational and Technical Educrtion 

20 - Soci? 1 Science Education 

21 - Mus Art and Humanities Education 



28-29 



ERIC Subject of Requesc 



Appendix C 

Cross Classification Tables 
for Requests Made to the Retrieval Center 
1967 - 1970 



Tables 24-30 
Tables 31 -38 

Tables 39-46 
Tables 47-51 
Tables 52-56 

Tables 57-61 

Tables 62-66 

Tables 67-71 



Cross Classification by Colorado School Districts 



Cross Classification by Six Participating States 
Outside Colorado 



Cross Classification by Date of Request 

Cross Classification by ERIC Subject Code: Reading 

Cross Classification by ERIC Subject Code: Educational 
Administration 



Cross Classification by HFRL Subject Code: Curriculum 

Development 

Cross Classification by HFRL Subject Code: Instructional 

Procedure and Techniques 

Cross Classification by HFRL Subject Code: Administrative 

Procedure and Organization 



Table 24 

Number of Requests by Occupation or Title 
in Colorado School Districts 
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Table 25 

Number of Requests by Institution 
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Number of Requests by Date of Request 
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Table 27 

Number of Requests by Type of Profile Sent 
in Colorado School Districts 
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Number of Requests by HFRL Subject 
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Number of Requests by Date of Request in 6 States 
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Date of Request 
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Prior to 1968 


0 


0 


0 
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January to June 1968 
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July to December 1968 
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1 
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0 
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July to December 1969 
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6 


89 


1 
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January to June 1970 
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11 
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7 


1 


3 


July to December 1970 


33 


32 


112 


342 


58 


219 


Totals 


51 


50 


307 


352 


62 


222 


Number of Requests 


by Type of 


Profi le 


i Sent in 


i 6 States 




Type of Profile Sent 














Current Interest only 
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11 


93 


180 


40 


128 


History only 
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8 


1 
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Manual only 
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16 


4 
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Current Interest & History 
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98 


11 
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Current Interest & Manual 
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4 


13 


History and Manual 
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Current Interest, History 
and Manual (all three) 
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4 
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20 


1 


7 
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51 


49 
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344 


61 
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Evaluation of the Information Retrieval Center, Boulder, Colorado 
Responses to Evaluation Questionnaire in 6 States 

Question 1: Were the abstracts in this profile pertinent to your request? 



Response 




& 

& 


<3* 

/ 




<S 




-i nno/ 
1 UU/o 


4 1 


1 


0 


9 


1 


4 


75% 


4 


1 


12 


26 


5 


13 


50% 


0 


0 


1 


7 


1 


8 


25% 


0 


0 


3 


18 


5 


10 


0% 


0 


0 


0 


JL 


2 


5 


Totals 


8 


2 


16 


62 


14 


40 



Question 2: Were the Manual search materials (zeroxed articles, 

bibliography, pamphlets or books) in the profile pertinent? 



100% 


1 


4 


7 


6 


2 


75% 


2 


7 


3 


0 


5 


50% 


0 


0 


1 


0 


1 


25% 


0 


1 


1 


0 


4 


0% 


__0 


0 


6 


]_ 


0 


Total s 


3 


12 


18 


7 


12 
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Table 36 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

Responses to Evaluation Questi onnai re in 6 States 

Question 3: Would you be willing t- pay for educational information retrieval 

services on a Cost-Per-Profi le basis? 









Response 


1 4k 




<■? 




/ 


S3 


Yes 


6 


3 


11 


51 


15 


28 


No 


0 


0 


4 


& 


0 


6 


Additional 


Comments 


on Evaluation 


Form 






Category of Response 














Generally Positive 


4 




8 


15 


2 


7 


Positive with 














Reservati ons 
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5 


2 


2 


Negati ve 
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10 


1 


6 


Neutral 
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1_ 


6 


1 


4 


Totals 


4 




11 


36 


6 


19 
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ERIC 
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Number of Requests by State or District 
Sorted by Date of Request 
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Table 40 

Number of Requests by Occupation or Title 
Sorted by Date of Request 
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Table 41 

Number of Requests by Institution 
Sorted by Date of Request 
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Table 42 

Number of Requests by HFRL Subject 
Sorted by Date of Request 
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Number of Requests by ERIC Subject 
Sorted by Date of Request 
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Table 43 , . 

Number of Requests by ERIC Subject (Continued) 
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Mean Manual Processing Time 13.34 days 17.82 days 27.52 days 17.97 days 

(Standard Deviation) (18.173) (17.838) (19,502) (11.144) 



Table 45 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Responses to Evaluation Questionnaire by Date 
Question 1: Were the abstracts in this profile pertinent to your request? 

1969 1970 1970 

Response July to Dec Jan to June July to Dec 



100% 


3 


19 


42 


75% 


2 


43 


93 


50% 


0 


3 


43 


25% 


2 


19 


77 


0% 


0 


4 


12 




7 


88 


267 



Question 2: Were the Manual search materials (zeroxed articles, 

bibliography, pamphlets or books) in the profile pertinent? 



100% 


2 


34 


47 


75% 


0 


26 


36 


50% 


0 


1 


12 


25% 


2 


10 


15 


0% 


0 


4 


9 




4 


75 


119 



217 



207 



Table 4G 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Responses to Evaluation Questionnaire by Date 



Question 3: Would you be willing to pay for educational information 

retrieval on a Cost-Per-Profile basis? 



Response 



1969 

July to Dec 



1970 

Jan to June 



1970 

July to Dec 



Yes 

No 



13 77 205 

0 8 35 



Additional Comments on Evaluation Form 



Category of Response 

Generally Positive 

Positive with 
Reservati ons 

Negative 

Neutral 

Totals 



9 



34 



87 



0 

1 



7 



24 

26 



1 7 II 

11 53 154 




218 



208 



Table 47 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Reading Requests by State or District 

No. of 

Requests % State or District 



52 


35.8 


1 . 


0 


0 


2. 


1 


0.7 


3. 


2 


1 ,4 


4. 


5 


3.4 


5. 


2 


1.4 


6. 


0 


0 


7. 


20 


13.8 


8. 


1 


0.7 


9. 


12 


8.3 


10. 


20 


13.8 


11. 


3 


2.1 


12. 


17 


11.7 


13. 


1 


0.7 


14. 


9 


6.2 


15. 


145 


100.0 





ERIC 



Boulder Valley Public Schools, RE-2J 

Westminster, Adams 5Q 

Thorn ton-Northgl enn , Eastlake, Adams 12 

Loveland (Big Thompson), Larimer R-2J 

Ft. Collins, Poudre, Lsvimer R-l 

Longmont, St. Vrain Valley, Boulder RE-1J 

Estes Park, Larimer R-3 

Colorado not in NCBOCES 

Nevada 

South Dakota 
Washington 
Oregon 
Utah 
Wyomi ng 

All other States 



Total 



209 



V: Table 48 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Reading Requests by Occupation or Title 



No. of 
Requests 


% 


Occupation or Title 


1 


0.78 


1 . 


Student 


23 


17.82 


2. 


Teacher, Elementary 


7 


5.43 


3. 


Teacher, Secondary 


3 


2.33 


4. 


Teacher, College or University 


8 


6.20 


5 . 


Librarian 


63 


48.83 


6. 


Specialists, Coordinators, Directors 
and Consultants 


4 


3.10 


7. 


Pupil Personnel - Counselor, Psychologist 


9 


6.98 


8. 


Administrator - Principal 


7 


5.43 


9. 


Administrator - Superintendent 


2 


1.55 


10. 


Layman 


0 


0 


11 . 


Other 


2 


1.55 


12. 


Teacher, Special Education 


129 


100.00 




Total 






ERIC 



220 



210 



Table 49 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Reading Requests by Institution 



No. of 
Requests 


% 


Institution 


42 


30.21 


1 . 


Elementary School 


17 


12.23 


2. 


High School, Junior High School 


5 


3.60 


3. 


College or University 


10 


7.19 


4. 


State Department of Education 


2 


1 .44 


5. 


Research Organization 


52 


37.41 


6. 


District Administrative Unit 


0 


0 


7. 


Teacher Training Institution 


8 


5.76 


8. 


ERIC and Other Resource Centers 


3 


2.16 


9. 


Other 


0 


0 


10. 


Special Education Institution 


139 


99.73 




Total 



ERIC 






Table 50 



211 



Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Reading Requests by Date of Request 



No. of 
Requests 


% 


Date of Request 


0 


0.00 


Prior to 1968 


2 


1.39 


January to June 1968 


2 


1.39 


July to December 1968 


15 


10.42 


January to i ne 1969 


19 


13.19 


July to ' ;emb r 1969 


27 


1 8.75 


January to Ji ne 1970 


79 


54.87 


July to December 1970 



Mean Computer Processing Time 17.82 days 

(Standard Deviation) (13.675) 

Mean Manual Processing Time 21.26 days 

(Standard Deviation) (18.519) 



Number of Reading Requests by Type of Profile Sent 

No. of 



Requests 


% 


Type of Profile 


53 


38.68 


Current Interest only 


2 


1.46 


History only 


14 


10.22 


Manual only 


15 


10.95 


Current Interest and History 


39 


28.47 


Current Interest and Manual 


0 


0.00 


History and Manual 


14 


10.22 


Current Interest, History and Manual 


137 


100.00 


Total 



ERIC 



Evaluation of Information Retrieval Center, Boulder, Colorado 

Number of Readi’ j Requests (ERIC Subject Code) 
by HFRL Subject Code 



No. of 



Reading 

Requests 


% 


HFRL Subject 


29 


19.73 


Curriculum Development 


19 


12.92 


Pupil Characteristics. 3e /ior aid Guidance 


74 


50.33 


Instructional Procedure f. Tec: Cliques 


4 


2.72 


Administrative Procedure and Organization 


0 


0.00 


Teacher Education, Pre and In-Service 


4 


2.72 


Subject Matter Topic 


10 


6.80 


Special Education 


7 


4.76 


Instructional Resources, Research 


0 


0.00 


Other 


147 


99.98 


Total 



223 



Table 52 



213 



Evaluation of the Information Retrieval Center, Boulder, Colorado 
Number of Requests by ERIC Subject Code 
Educational Administration 



No. of 

Requests % State or District 



310 


32.1 


1 . 


Boulder Valley Public Schools, RE-2J 


10 


1.0 


2. 


Westminster, Adams 50 


34 


3.5 


3. 


Thomton-Northglenn, Eastlake, Adams 12 


6 


.6 


4. 


Loveland (Big Thompson) Larimer R-2J 


20 


2.1 


5. 


Ft. Collins, Poudre, Larimer R-l 


25 


2.6 


6. 


Longmont, St. Vrain Valley, Boulder RE-10 


4 


.4 


7. 


Estes Park, Larimer R-3 


161 


16.7 


8. 


Colorado not In NCBOCES 


15 


1.6 


9. 


Nevada 


120 


12.4 


10. 


South Dakota 


135 


14.0 


11. 


Washington 


14 


1.4 


12. 


Oregon 


46 


4.8 


13. 


Utah 


23 


2.4 


14. 


Wyomi ng 


43 


4.5 


15, 


All other States 


966 


100.1 




Total 



ERIC 



224 



214 



Table 53 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

ERIC Subject Code: Educational Administration 

Number of Requests by Occupation or Title 



No. of 

Requests % Occupation or Title 



36 


4.1 


1 . 


Student 


57 


6.5 


2. 


Teacher, Elementary 


52 


5.9 


3. 


Teacher, Secondary 


32 


3.7 


4. 


Teacher, College or University 


75 


to 

CO 


5. 


Librari an 


342 


29.1 


6 . 


Specialists, Coordinators, Directors 
and Consultants 


19 


2.2 


7 . 


Pupil Personnel - Counselor, Psychologist, 
and Nurse 


131 


15.0 


8. 


Administrator - Principal 


92 


10.5 


9 . 


Administrator - Superintendent 


8 


.9 


10 . 


Layman 


26 


3.0 


11 . 


Other 


4 


.5 


12. 


Teacher, Special Education 


874 


100.0 




Total 



ERJC 



225 



3 



Table 54 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

ERIC Subject Code: Educational Administration 

Number of Requests by Institution 



No. of 

Requests % Institution 



174 


18.4 


1 . 


Elementary School 


168 


17.8 


2. 


High School, Junior High School 


94 


9.9 


3. 


College or University 


70 


7.4 


4. 


State Department of Education 


37 


3.9 


5. 


Research Organization 


300 


31.7 


6. 


District Administrative Unit 


3 


.3 


7. 


Teacher Training Institution 


65 


6.9 


8. 


ERIC and Other Resource Centers 


31 


3.3 


9. 


Other 


3 


.3 


10. 


Special Education Institution 


945 


99.9 




Total 



ERIC 



226 



Evaluation of the Information Retrieval Center, Boulder, Colorado 



ERIC Subject Code: Educational Administration 



Number of Requests by Date of Request 
No. of 

Requests % Date of Request 



0 


0 


Prior to 1968 


32 


3.3 


January 


to June 1968 


22 


2.3 


July to 


December 1968 


69 


7.2 


January 


to June 1969 


162 


16.8 


July to 


December 1969 


203 


21.1 


January 


to June 1970 


475 

SSI 


49.3 

100.0 


July to 


December 1970 


Mean Computer Processing Time 
(Standard Deviation) 


17.79 

(11.407) 


Mean Manual 
(Standard 


Processing Time 
Deviati on) 


19.31 

(14.063) 


Number of Requests 


by Type 


of Profile Sent 



No. of 



Requests 


% 


Type of Profile 


310 


35.1 


Current Interest only 


14 


1.6 


History only 


87 


9.8 


Manual only 


122 


13.8 


Current Interest and History 


259 


29.3 


Current I )rest and Manual 


3 


.3 


History and Manual 


89 


10.1 


Current Interest, History and Manual 


884 


100.0 


Total 



227 



Table 56 



217 



Evaluation of the Information Retrieval Center, Boulder, Colorado 

ERIC Subject Code: Educational Administration 

Number of Requests by HFRL Subject Code 



Requests % Subject of Requests 



34 


3.5 


1. Curriculum Development 


2 


.2 


2. Pupil Characteristic^ , Behavior, Guidance 


215 


22.0 


3. Instructional Procedure and Techniques 


716 


73.2 


4. Administrative Procedure and Organization 


2 


.2 


5. Teacher Education, Pre and In-Service 


3 


.3 


6. Subject Matter Topic 


0 


0 


7. Special Education 


6 


.6 


8. Instructional Resources, Research 


978 


100.0 


Total 




O 

ERLC 



228 



Table 57 



218 



Evaluation of the Information Retrieval Center, Boulder, Colorado 

HFRL Subject Code: Curriculum Development 

Number of Requests by State or District 



No. of 
Requests 


% 


State or District 


147 


37.8 


1. 


Boulder Valley Public Schools, RE-2J 


1 


.2 


2. 


Westminster, Adams 50 

1 


13 


3.3 


3. 


Thornton-Northglenn, Eastlake, Adams 12 


1 


.2 


4. 


Loveland (Big Thompson), Larimer R-l 


16 


4.1 


5. 


Ft. Collins, Poudre, Larimer R-l 


11 


2.8 


6. 


Longmont, St. Vrain Valley, Boulder RE-1J 


5 


1.2 


7. 


Estes Park, Larimer R-3 


42 


10.8 


8. 


Colorado not in NCBOCES 


1 


.2 


9. 


Nevada 


28 


7.2 


10. 


South Dakota 


39 


10. C 


11. 


Washington 


6 


1.5 


12. 


Oregon 


50 


12.8 


13. 


Utah 


9 


2.3 


14. 


Wyoming 


19 


4.8 


15. 


All other States 


388 


99.2 




Total 



ERIC 



229 



219 



Table 58 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

HFRL Subject Code: Curriculum Development 

Number of Requests by Occupation or Title 



No. of 

Requests % Occupation or Title 



8 


2 .1 


1 . 


Student 


34 


9.2 


2. 


Teacher, Elementary 1 


76 


20.5 


3. 


Teacher, Secondary 


14 


3.7 


4. 


Teacher, College or University 


20 


5.4 


5. 


Librarian 


143 


38.7 


6. 


Specialists, Coordinators, Directors 
and Consultants 


12 


3.2 


7. 


Pupil Personnel - Counselor, Psychologist, 
and Nurse 


26 


7.0 


8. 


Administrator - Principal 


24 


6.5 


9. 


Administrator - Superintendent 


3 


.8 


10. 


Layman 


6 


1.6 


11. 


Other 


3 


.8 


12. 


Teacher, Special Education 


369 


99.5 




Total 







230 



220 



Table 59 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

HFRL Subject Code: Curriculum Development 

Number of Requests by Institution 



No. of 

Requests % Insti tution 



53 


13.7 


1 . 


Elementary School 


111 


28.8 


2. 


High School, Junior High School 


33 


8.5 


3. 


College or University 


30 


7.7 


4. 


State Department of Education 


9 


2.3 


5. 


Research Organization 


95 


24.6 


6. 


District Administrative Unit 


1 


.2 


7. 


Teacher Training Institution 


38 


9.8 


8. 


ERIC and Other Resource Centers 


13 


3.3 


9. 


Other 


2 


.5 


10. 


Special Education Institution 


385 


99.4 




Total 



O 




931 



Table 60 

Evaluation of the Information Retrieval Center, Boulder, Colorado 



221 



HFRL Subject Code: Curriculum Development 

Number of Requests by Date of Request 



No. of 
Requests 

0 

15 

14 

23 

49 

61 

228 



% 



0 

3.8 
3.6 

5.9 
12.6 
15.6 
58.5 

loO - 



Date of Request 
Prior to 1968 
January to June 1968 
July to December 1968 
January to June 1969 
July to December 1969 
January to June 1970 
July to December 1970 



Mean Computer Processing Time 19.24 days 

(Standard Deviation) (11.785) 

Mean Manual Processing Time 23.85 days 

(Standard Deviation) (23.179) 



No. of 
Requests 



Number of Requests by Type of Profile Sent 



Type of Profile 



147 


41.1 


Current Interest only 


8 


2.2 


History only 


26 


7.2 


Manual only 


79 


22.1 


Current Interest and History 


63 


17.6 


Current Interest and Manual 


0 


0 


History and Manual 


34 


9.5 


Current Interest, History and 


357 


99.7 


Total 






232 




Table 61 



222 



Evaluation of the Information Retrieval Center, Boulder, Colorado 
HFRL Subject Code: Curriculum Development 

Number of Requests by ERIC Subject Code 
No. of 



3 

ERIC 



quests 


% 


Subject of Request 


0 


0 


1. 


Adult Education 


1 


.2 


2. 


Counseling and Student Personnel Services 


21 


5.2 


3. 


Disadvantaged 


14 


3.5 


4. 


Early Childhood Education 


34 


8.5 


5. 


Educational Administration 


12 


3.0 


6. 


Educational Media and Technology 


2 


.5 


7. 


Exceptional Children 


3 


.7 


8. 


higner Education 


1 


.2 


9. 


Junior Colleges 


10 


2.5 


10. 


Library and Information Services 


4 


1.0 


n. 


Linguistics 


29 


7.2 


12. 


Reading 


0 


0 


13. 


Rural Education and Small Schools 


57 


14.2 


14. 


Science and Mathematics Education 


4 


1.0 


15. 


Teacher Education 


39 


9.7 


16. 


Teaching of English 


6 


1.5 


17. 


Teaching Foreign Languages 


9 


2.2 


18. 


Tests, Measurements and Evaluation 


48 


12.0 


19. 


Vocational and Technical Education 


81 


20.3 


20. 


Social Science Education 


23 


5.7 


21. 


Music, Art and Humanities Education 


399 


99.1 




Total 



223 



Table 62 

Evaluation of the Information Retrieval Center, Boulder, Colorado 



HFRL Subject Code: Instructional Procedure and Techniques 

Number of Requests by State or District 



No, of 

Requests % State or District 



217 


36.2 


1 . 


7 


1.1 


2. 


11 


1.8 


3. 


2 


.3 


4. 


18 


3.0 


5. 


7 


1.1 


6. 


2 


.3 


7. 


79 


13.1 


8. 


12 


2.0 


9. 


65 


10.8 


10. 


84 


14.0 


11. 


15 


2.5 


12. 


56 


9.3 


13. 


5 


.8 


14. 


19 


3.1 


15. 


599 


99.4 





ERIC 



Boulder Valley Public Schools, RE-2J 
Westminster, Adams 50 
Thornton-Northglenn, Eastlake, Adams 12 
Loveland (Big Thompson), Larimer R-l 
Ft. Collins, Poudre, Larimer R-l 
Longmont, St. Vrain Valley, Boulder RE-1J 
Estes Park, Larimer R-3 
Colorado not in NCBOCES 
Nevada 

South Dakota 

Washington 

Oregon 

Utah 

Wyoming 

All other States 
Total 



234 



224 



Table 63 

Evaluation of the Information Retrieval Center, Boulder. Colorado 
HFRL Subject Code: Instructional Procedure and Techniques 

Number of Requests by Occupation or Title 

No. of 



Requests 


% 


Occupation or Title 


17 


3.1 


1 . 


Student 


69 


12.7 


2. 


Teacher, Elementary 


84 


15.5 


3. 


Teacher, Secondary 


20 


3.7 


4. 


Teache*', College or Un*’ ersity 


47 


8.7 


5. 


Librarian 


200 


37.0 


6. 


Specialists, Coordinate' , Directors 
and Consultants 


11 


2.0 


7. 


Pupil Personnel - Counselor, Psychologist 
and Nurse 


52 


9.6 


8. 


Administrator - Principal 


32 


5.9 


9. 


Administrator - Superintendent 


2 


.3 


10. 


Layman 


3 


.5 


11. 


Other 


3 


.5 


12. 


Teacher, Special Education 


540 


99.5 




Total 



ERIC 






225 



Table 54 

Evaluation of the Information Retrieval Center, Boulder, Colorado 
HFRL Subject Code: Instructional Procedure and Techniques 

Number of Requests by Institution 



No. of 

Requests % Institution 



147 


24.7 


1 . 


Elementary School 


1 5 


24.3 


• 2. 


High School, Junior High School 


53 


8.9 


3. 


College or University 


^0 


6.7 


4. 


State Department of Education 


19 


3.1 


5. 


Research Organization 


156 


26.2 


6. 


District Administrative Unit 


3 


.5 


7. 


Teacher Training Institution 


22 


3.6 


8. 


ERIC and Other Resource Centers 


9 


1.5 


9. 


Other 


1 


.1 


10. 


Special Education Institution 


595 


99.6 




Total 



O 

ERIC 



236 



Evaluation of the Information Retrieval Center, Boulder, Colorado 



HFRt Subject Code: Instructional Procedures and Techniques 

Number of Requests by Date of Request 



No. of 
Requests 


% 


Date of Request 


1 


.1 


Prior to 1968 


33 


5.4 


January to lune 1968 


21 


3.4 


July to December 1968 


47 


7.7 


January to June 1969 

j 


78 


12.8 


July to December '1969 


93 


15.3 


January t: June 1970 


334 


55.0 


July to December 1970 


Wf 


99.7 





Mean Computer Processing Time 
(Standard Deviation) 

Mean Manual Processing Time 
(Standard Deviation) 



18-44 davs 
(10.639) 

21 .87 days 
(16.155) 



Number of Requests by Type of Profile Sent 

No. of 



Requests 


% 


Type of Profile 


219 


39.1 


Current Interest only 


10 


1.7 


History only 


62 


11.0 


Manual only 


80 


14.2 


Current Interest and History 


150 


26.7 


Current Interest and Manual 


2 


.3 


History and Manual 


37 


6.6 


Current Interest, History and Manual 


560 


99.6 


Total 



237 



Table 66 



22 ? 



Evaluation of the Information Retrieval Center, Boulder, Colorado 






HFRl- Subject Code Instructional Procedures and T 'echr ques 
Number of Requests by ERIC Subject Code 
No. of 



Requests 


% 


Subject of Request. 


1 


4 0 


1. 


Adult Education 


14 


2.3 


2. 


Counseling and Student Personnel Services 


7 


1.1 


3. 


Disadvantaged 


5 


A 

.a 


4. 


Early Childhood Educatior 


215 


34.3 


5. 


Educational Administration 


44 


7.1 


6. 


Educational Media and Technology 


6 


1.0 


7. 


Exceptional Children 


2 


.3 


8. 


Higner Education 


1 


.2 


9. 


Junior Colleges 


34 


5.5 


10. 


Library and Information Services 


3 


.5 


11. 


Linguistics 


74 


12.0 


12. 


Reading 


1 


.2 


13. 


Rural Education and Small Schools 


69 


11.2 


14. 


Science and Mathematics Education 


6 


1.0 


15. 


Teacher Education 


27 


4.4 


16. 


Teaching of English 


8 


1.3 


17. 


Teaching Foreign Languages 


16 


2.6 


18. 


Tests, Measurements and Evaluation 


26 


4.2 


19. 


Vocational and Technical Education 


44 


7.1 


20. 


Social Science Education 


15 


2.4 


21. 


Music, Art and Humanities Education 


618 


100.2 




Total 






228 



Table 67 

Evaluation of the- Information Retrieval Center, Boulder, Colorado 
HFRL Subject ' Je Administrative Procedure and Organization 

Number of Requests by State or District 

iMo . of 

Requests % State or District 



249 


3D 1 


1 . 


Boulder Valley Public Schools, RE-2J 


7 


£ 


2. 


Westminster, Adams 50 


29 


3.5 


3. 


Thorn ton-Northgl enn , Eastlake, Adams 12 


5 


.5 


4. 


Loveland (Big Thompson), Larimer R-l 


14 


1.6 


5. 


Ft. Collins, Poudre, Larimer R-l 


22 


2.6 


6. 


Longmont, St. Vrain valley, Boulder RE-1J 


4 


.4 


7. 


Estes Park, Larimer R-3 


140 


16.9 


8. 


Colorado not in NCBOCES 


13 


1 .5 


9. 


Nevada 


109 


13.1 


10. 


South Dakota 


117 


14.1 


11. 


Washington 


17 


2.0 


12. 


Oregon 


38 


4.6 


13. 


Utah 


22 


2.6 


14. 


Wyoming 


40 


4.8 


15. 


All other States 


826 


99.2 




Total 




239 



229 



Table 68 

Evaluation c" nformation Retrieval Center, Boulder, Colorado 

HFRL Subject :ie: Administrative Procedure and Organization 



i:„ ■ of Requests by Occupation or Title 



No. of 

Requests ' Occupation or Title 



31 




1. 


Student 


34 




7 

■— • 


« wvi v* W * 9 i VV% 1 J 


35 


i- j 


3 . 


Teacher, Secondary 


27 


3.6 


4. 


Teacher, College or University 


77 


10. - 


5. 


Librarian 


304 


40.6 


6 . 


Specialists, Coordinators, Directors 
and Consultants 


14 


1.8 


7. 


Pupil Personnel - Counselor, Psychologist 
and Nurse 


111 


14. E 


8. 


Administrator - Principal 


81 


10.8 


9. 


Administrator - Superintendent 


7 




10. 


Layman 


25 


1,3 


11. 


Other 


1 


J 


12. 


Teacher, Special Education 


747 


99.4 




Total 



* 




240 



230 



I able 69 

Evaluation of the Information Retrieval Center, Boulder, Colorado 

HFRL Subject Code: Administrative Procedure and Organization 

Number of Requests by Institution 



No. of 

Requests %_ 



Institution 



129 


16.0 


1 . 


Elementary School 


138 


17.2 


2. 


High School, Junior High School 


83 


10.3 


3. 


College or University 


65 


8.1 


4. 


Slate Department of Education 


30 


3.7 


5. 


Research Organization 


262 


32.6 


6. 


District Administrative Unit 


1 


.1 


7. 


Teacher Training Institution 


65 


8.1 


8. 


ERIC and Other Resource Centers 


29 


3.6 


9. 


Other 


1 


.1 


10. 


Special Education Institution 


804 


99.8 




Total 



O 

ERiC 
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Table 70 231 

Evaluation of the Information Retrieval Center, Boulder, Colorado 



HFRL Subject Code: Administrative Procedure and Organization 

Number of Requests by Date of Request 



No. of 
Requests 


% 


Date of 


Request 


0 


0 


Prior to 1968 


16 


1.9 


January 


to June 1968 


16 


1.9 


July to 


December 1968 


59 


7.1 


January 


to June 1969 


142 


17.3 


July to 


December 1969 


175 


21.3 


January 


to June 1970 


411 

FT5T 


50.1 


July to 


December 1970 


Mean Computer Processing Time 
(Standard Deviation) 


17.67 days 
(11.711) 


Mean Manual Processing Time 
(Standard Deviation) 


18.82 days 
(13.842) 



Number of Requests by Type of Profile Sent 



No. of 
Requests 


% 


Type of Profile 


268 


35.0 


Current Interest only 


9 


1.1 


History only 


30 


1C. 4 


Manual only 


103 


13.4 


Current Interest and History 


221 


28.9 


Current Interest and Manual 


3 


.3 


History and Manual 


80 


10.4 


Current Interest, History and Manual 


764 


99.9 


Total 







0/10 



Table 71 
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Evaluation of the Information Retrieval Center, Boulder, Colorado 
HFRL Subject Code: Administrative Procedure and Organization 



Number 


of Requests 


by ERIC Subject Code 


No. of 
Requests 


% 


Subject of Request 


1 


.1 


1. Adult Education 


3 


.4 


2. Counseling and Student Personnel Services 


1 


.1 


3, Disadvantaged 


4 


.5 


j 

4. Early Childhood Education 


715 


86.0 


5. Educational Administration 


20 


2.4 


6. Educational Media and Technology 


1 


.1 


7. Exceptional Children 


8 


1.0 


8. Higner Education 


7 


.8 


9. Junior Colleges 


33 


4.0 


10. Library and Information Services 


0 


0 


11. Linguistics 


4 


.5 


12. Reading 


6 


.7 


13. Rural Education and Small Schools 


4 


.5 


14. Science and Mathematics Education 


0 


0 


15. Teacher Education 


2 


.2 


16. Teaching of English 


1 


.1 


17. Teaching Foreign Languages 


8 


1.0 


18. Tests, Measurements and Evaluation 


9 


1.1 


19. Vocational and Technical Education 


3 


.4 


20. Social Science Education 


2 


.2 


21. Music, Art and Humanities Education 


ERJC 833 


100.1 


Total 
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User Questionnaire 
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Evaluation Questionnaire 
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'ION I Please respond to the following items by checking in the appropriate space or writing your 
response in the space provided. You will not be asked to identify yourself. 

Please check the main purpose(s) for which you used the services of the Resources Center. 



1. 


General knowledge about subject area 


6. 


Preparation of a speech or report 


2. 


Research 


7. 


Methods of classroom instruction 


3. 


Assignments and term papers 


8. 


School administration problems 


4. 


Preparation or updating of 


9. 


Other (explain) 


course bibliographies 






5. 


Curriculum revision 






* or 


how did you first hear about the services 


of the 


Resource Center? 



1. Work associates 

2. Newspaper 



3. Bulletins and newsletters 

4. Staff meeting 



5. Presentation by Resource 

Center Personnel 

6. Other 



How useful have you found the information provided by the Information Retrieval Center? 



1 . Not Useful 



3. Moderately 
Useful 



4. 



5. Very Useful 



Have the abstracts in the -profiles (computer) and the manual search materials sent you usually been 



pertinent to your request? 



Computer abstracts: 
Manual search: 



^ es 

yes 



j\ o 
no 



_not applicable 
not applicable 



Which one of the following statements best describes the Information you have received from the Center? 

K . All the information I received would not have otherwise been available. 

2. Most of the information I received would not have otherwise been available, 

3. Some of the information i received would not have otherwise been available* 

4. Essentially, all of the infonnation I received would have otherwise been available. 



Which of the following materials or equipment are most lacking in your work situation? 

1. Microfiche reader 4. Government reports 



Microfiche reader 

2. Microfiche reader-printer 

3. Microfiche duplicator 



5. Adequate library facilities close by 



Which, if any, of the following problems have you encountered in using the services of the Resources 



Center? 

1 . 


Profile too general 


6 . 


Microfiche reader not readily available 


2. 


Too much information to wade through 


7. 


Took too long to process request 


3. 


Information not pertinent to request 


8. 


Confusion as to what kind of informal c 


4. 


Too little information 


9. 


the Center can provide 
No problems were encountered 


5. 


Complete documents (hardcopy or 
microfiche) not available 


10. 


Other (explain) 



Rank the following suggestions for improving the Center's services by placing "1 ' beside the most 
needed, "2" beside the next most, eic. , and "5" beside the least needed improvement. 



1. More specific request forms 

2. More accurate computer search 

3. Shorten the processing time of requests 



4. Provide more expanded subject area coverage 

5. Keep expanding and updating the data base 



Any other suggestions? 



o 

ERIC 
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luation Questionnaire 2 

Can you cite any instances where information from the Resources Center has contributed to changes in 
programs and practices? 



TION II. This scale has been prepared so that you can indicate how you feel about the Information 
Retrieval Center. Please respond to every item. In each case, draw a circle around the letter 
which represents your own reaction as follows: 



SA 

A 

N 

0 

SO 



if you strongly agree with the statement 

if you agree but not strongly so 

if you are neutral or undecided 

if you disagree brt not strongly Co 

if you strongly disagree with the statement 



The Retrieval Center has been a real help to me 


. SA 


A 


N 


D 


SO 


I feel the Federal Government is not justified in supporting the cost of 
the Retrieval Center 


. SA 


A 


N 


0 


SD 


The Retrieval Center should be continued and expanded. . . . • . 


. SA 


A 


N 


D 


SO 


The Retrieval is of questionable value 


. SA 


A 


N 


0 


SO 


The Idea behind the Retrieval Center is a good one 


. SA 


A 


N 


D 


SD 


Children are benefiting because of the existence of the Retrieval Center. 


. SA 


A 


N 


D 


SD 



ION III. 

Please check the geographic area where you work. 

Within Colorado: 

J. Boulder Valley District RE-2J 

2. Westminster, Adams 50 

J3. Thorn ton-Northglenn , EarUa^fc,. Adams 12 

4. Loveland, Big Thompson, Li. 'inter R-2J 

5. Fort Collins, Poudre, Larimer R-l District 

6. Longmont, St. Vrain Valley, Boulder RE-1J 

7. Estes Park, Larimer R-3 District 

8. Other towns and school districts within 
Co^raio but not listed above 

Please check the occupation category which best describes your present position. 

1. Student 

| 2. Teacher, Elementary 

3. Teacher, Secondary 

4. Teacher, college or university 

J>. Librarian 

6. Specialists, coordinators, directors, 
consultant, research specialist 

7. Pupil personnel--counselor , psychologist, school nurse 

Please check the number of years of professional experience you have had. 



i. 


Less than 5 years 


4. 


15-19 years 


2. 


5-9 years 


5. 


20-24 years 


3. 


10-14 years 


6 . 


More than 25 years 



8. Adninistrator - principal 

9. Administrator - superintendent 

10. Laymen 

11. Other 

12. Teacher - special education 



Other States Outside Colorado: 



9. 


Nevada 


10. 


South Dakota 


11. 


Washington 


12. 


Oregon 


13. 


Utah 


14. 


Wyomi ng 


15. 


Other states not listed above 



O 

ERIC 
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Appendix E 



Cross Classification Tables 
for User Questionnaire 

Tables 72-80 - Cross Classification Tables by Geographic Area 

Tables 81-89 - Cross Classification Tables by Occupation Category 

Tables 90-98 - Cross Classification Tables by Years of Professional 

Experience 
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Table 72 

Responses to User Questionnaire by Geographic Area 

Question 1: Please check the main purpose(s) for which you used the services 

of the Resources Center. 



Response 

1. General knowledge about subject area 

2. Research 

3. Assignments and term papers 

4. Preparation or updating of 

course bibliographies 

5. Curriculum revision 

6. Preparation of a speech or report 

7. Methods of classroom instruction 

8. School Administration problems 
9„ Other 

Total 



Colorado 
f % 


Outside 

_f 


Colorado 

% 


27 


18.7 


25 


14.9 


38 


26.4 


34 


20.2 


8 


5.6 


5 


3.6 


4 


2.8 


5 


3.0 


22 


15.3 


25 


14.9 


8 


5.6 


9 


5.4 


19 


13.2 


31 


18.5 


13 


9.0 


19 


11.3 


5 


3.5 


14 


8.3 


144 


100.1 


168 


100.1 




248 



238 



Table 73 

Responses to User Questionnaire by Geographic Area 

Question 2: Where or how did you first hear about the services of 

the Resource Center? 





Response 


Colorado 
f % 


Outside 


Colorado 

% 


1 . 


Work associates 


31 


34.1 


26 


25.0 


2. 


Newspaper 


3 


3.3 


1 


1.0 


3. 


Bulletins and Newsletters 


4 


4.4 


11 


10.6 


4. 


Staff meeting 


14 


15.4 


8 


7.7 


5. 


Presentation by Resource 
Center Personnel 


31 


34.1 


29 


27.9 


6. 


Other 


8 


8.8 


29 


27.9 




Total 


91 


100.1 


104 


100.1 
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Table 74 

Responses to User Questionnaire by Geographic Area 

Question 3: How useful have you found the information provided by the 

Information Retrieval Center? 





Response 


Colorado 
f % 


Outside 

_f 


Colorado 

% 


1 

3 • 


Not useful 


6 


8.3 


7 


7.7 


2. 




3 


4.2 


14 


15.4 


3. 


Moderately useful 


20 


27.8 


22 


24.2 


4. 




17 


23.6 


24 


26.4 


5, 


Very useful 


26 


36.1 


24 


26.4 




Total 


72 


100.0 


91 


100.1 



Mean Response 


3.750 


3.484 


Standard Deviation 


1.222 


1.244 



ERiC 
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Tab»e 75 

Responses to User Questionnaire by Geographic Area 

Question 4: Have the abstracts in the profiles (computer) and the 

manual search materials sent you usually been pertinent 
to your request? 



Computer Abstracts 



Response 


Colorado 
f % 


Outside 


Colorado 

% 


Yes 


57 


87.7 


61 


79.2 


No 


6 


9.2 


9 


11.7 


Not Applicable 


2 


3.1 


7 


9.1 


Total 


65 


100.0 


77 


100.0 



Manual Search Materials 



Response 


Colorado 
f % 


Outside 


Col orado 

% 


Yes 




81.4 


38 


61.3 


No 


2 


3.4 


5 


8.1 


Not Applicable 


9 


15.3 


19 


30.6 


Total 


59 


100.1 


62 


100.0 
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Responses to User Questionnaire by Geographic Area 
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Table 77 

Responses to User Questionnaire by Geographic Area 

Question 6: Which of the following materials or equipment are 

most lacking in your work situation? 



Colorado Outside Colorado 



Response 


__f 


% 




% 


Microfiche reader 


20 


17.4 


34 


24.6 


Microfiche reader-printer 


26 


22.6 


26 


18.8 


Microfiche duplicator 


25 


21.7 


33 


23.9 


Government Reports 


24 


20.9 


16 


11.6 


Adequate library facilities 
close by 


20 


17.4 


29 


21.0 


Total 


115 


100.0 


138 


99.9 
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Table 78 

Responses to User Questionnaire by Geographic Area 

Question 7: Which, if any, of the following problems have you encountered in 

using the services of the Resources Center? 



Response 


Col orado 
f % 


Outside 

__f 


Col orado 

% 


Profile too general 


15 


14.4 


15 


10.5 


Too much information to wade through 


7 


6.7 


6 


4.2 


Information not pertinent to request 


12 


11.5 


22 


15.4 


Too little information 


11 


10.6 


13 


9.1 


Complete documents (hardcopy or 
microfiche) not available 


4 


3.8 


13 


9.1 


Mi crof i che reader not readi ly avai 1 abl e 


10 


9.6 


16 


11.2 


Took too long to process request 


9 


8.7 


23 


16.1 


Confusion as to what kind of information 
the Center can provide 


8 


7.7 


9 


6.3 


No problems were encountered 


26 


25.0 


17 


11.9 


Other 


2 


1.9 


9 


6.3 


Total 


104 


9^ 9 


143 


100.1 



O 
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Table 79 

Responses to User Questionnaire by Geographic Area 



Question 8: Rank the following suggestions for improving the Center's services by placing "1" beside the 
most needed, ,, 2" beside the next most, etc., and "5" beside the least needed improvement. 



Colorado 

Rank: 1 2 3 4 5 Mean 



Response 


— 


% 


_f 


More specific request forms 
(N=47) 


n 


23.4 


4 


More accurate computer search(N=44) 


8 


18.2 


4 


Shorten the processing time of 
requests (N=47) 


6 


12.8 


9 


Provide more expanded subject 
area coverage (N=53) 


19 


35.8 


16 


Keep expanding and updating 
the data base (N=59) 


27 


45.8 


15 



% 




% 


_f 


% 


_f 


% 


Rank 


8.5 


9 


19.1 


8 


17.0 


15 


31.9 


3.255 


9.1 


7 


15.9 


17 


38.6 


a 


18.2 


3.295 


19.1 


9 


19.1 


8 


17.0 


15 


31.9 


3.362 


33.2 


14 


26.4 


1 


1.9 


3 


5.7 


2.113 


25.4 


5 


8.5 


8 


13.6 


4 


6.8 


2.102 



Outside Colorado 



Rank: 




1 




2 




3 




4 




5 


Mean 


Response 


JF 


% 


Jf 


% 


Jf 


% 


jf 


% 


Jf 


V 

to 


Rank 


More specific request forms 
(N=62) 


14 


22.6 


8 


12.9 


13 


21.0 


12 


19.4 


15 


24.2 


3.097 


More accurate computer search 
(N=56) 


6 


10.7 


6 


10.7 


7 


12.5 


16 


28.6 


21 


37.5 


3.714 


Shorten the processing time of 
requests (N=65) 


25 


38.5 


10 


15.4 


10 


15.4 


10 


15.4 


10 


15.4 


2.538 


Provide more expanded subject 
area coverage (N=61) 


17 


27.9 


17 


27.9 


14 


23.0 


9 


14.8 


4 


6.6 


2.443 


Keep expanding and updating 
the data base (N=62) 


26 


41.9 


19 


30.6 


8 


12,9 


5 


8.1 


4 


6.5 


2.065 
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Table 80 

Response to User Questionnaire by Geographic Area 
Attitude Scale 
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Colorado 









No. 


of Responses 




Mean 




ftweiiiaii 


SAU1 


mi 


Mill 


mi 


SfilSl 


Score. 


10. 


The Retrieval Center has been a real help 


26 


30 


9 


5 


1 


1.944 


12. 


The Retrieval Center should be continued 


44 


24 


2 


0 


1 


1.451 


14. 


The idea behind the Retrieval Center is 


52 


16 


2 


0 


1 


1.338 


15. 


Children are benefiting because of the 
existence of the Retrieval Center 


19 


24 


24 


2 


1 


2.171 






$A(5) 


mi 


m. 1 


mi 


SD(1). 




11. 


I feel the Federal Government Is not 
justified in support ng the cost of the 
Retrieval Center 


2 


z 


7 


21 


39 


1.690 


13. 


The Retrieval Is of questionable value . . . 


0 


5 


5 


18 


42 


1 .614 



Mean Total Attitude Score for Colorado Sample: 
Standard Deviation 



Outside Colorado 



Question 

10. The Retrieval Center has been a real help 
to me 

12. The Retrieval Center should be continued 
and expanded 

14, The Idea behind the Retrieval Center Is 

a good one 

15. Children are benefiting because of the 
existence of the Retrieval Center, . . . . 



11. I feel the Federal Government is not 

justified in supporting the cost of the 
Retrieval Center. . . * * * * 

13. The Retrieval is of questionable value. . . 





No. 


of Response 


!S 




Mean 

Score 


5 5HT 






MT 


SLHbJ 


30 


33 


16 


6 


4 


2.112 


50 


27 


9 


3 


2 


1.681 


61 


20 


4 


3 


0 


1.420 


18 

SA(5) 


33 

A(4) 


32 

N(3) 


4 

0(2) 


2 

SD(1) 


2.315 


2 


6 


12 


28 


42 


1.867 


1 


5 


9 


34 


41 


1.789 




Mean Total Attitude Score for Outside Colorado: 
Standard Deviation 



11.161 

4.397 






Table 81 

Responses to User Questionnaire by Occupation Category 
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Question 1: Please check the main purpose(s) for which you used the services of 

the Resources Center. 







Administrators 


Non-Admi ni s trators 




Response 


f 


% 


f 


% 


1 . 


General knowledge about subject area 


25 


15.8 


26 


18.3 


2 . 


Research 


35 


22.2 


33 


23.2 


3 . 


Assignments and term papers 


3 


1.9 


10 


7.0 


4 . 


Preparation or updating of 
course bibliographies 


4 


2.5 


4 


2.3 


5 . 


Curriculum revision 


29 


18.4 


17 


12.0 


6 . 


Preparation of a speech or report 


8 


5.1 


9 


6.3 


7. 


Methods of classroom instruction 


19 


12.0 


30 


21.1 


8 . 


School Administration Problems 


24 


15.2 


8 


5.6 


9, 


Other 


11 


7.0 


5 


3.5 




Total 


158 


100.1 


142 


99.8 
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Table 82 

Responses to User Questionnaire by Occupation Category 



Question 2: Where or how did you 

Resource Center? 

Response 

1. Work associates 

2. Newspaper 

3. Bulletins and Newsletters 

4. Staff meeting 

5. Presentation by Resource 

Center Personnel 

6. Other 
Total 



fi rst 


hear about 


the services 


of the 


Administrators 


Non-Administrators 


f 


% 


f 


% 


30 


32.6 


23 


25.3 


1 


1.1 


3 


3.3 


g 


9.8 


5 


5.5 


8 


8.7 


13 


14.3 


26 


28.3 


32 


35.2 


18 


19.6 


15 


16.5 


92 


100.1 


91 


100.1 
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Table 83 

Responses to User Questionnaire by Occupation Category 



Question 3: How useful have you found the information provided by the 

Information Retrieval Center? 



Administrators Non-Administrators 



Response f 



1 . Not useful 2 

2. 10 

3. Moderately useful 15 

4. 27 

5. Very useful 25 

Total 79 

Mean Response 3.797 

Standard Deviation 1.095 



% f % 



2.5 


9 


12.2 


12.7 


7 


9.5 


19.0 


24 


32.4 


34.2 


12 


16.2 


31.6 


22 


29.7 


100.0 


74 


100.0 



3 419 
1 .325 
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Table 84 

Responses to User Questionnaire by Occupation Category 

Question 4: Have the abstracts in the profiles (computer) and the 

Manual search materials sent you usually been pertinent 
to your request? 



Computer Abstracts 

Admi ni s trators Non-Admi ni s trators 



Response 


f 


% 


f 


% 


Yes 


58 


86.6 


55 


83.3 


No 


5 


7.5 


9 


13.6 


Not Applicable 


4 


6.0 


2 


3.G 


Total 


67 


100.1 


66 


99.9 




Manual 


Search .materials 




Response 


Admi 

f 


nistrators 

% 


Non-Administrators 
f % 


Yes 


39 


70.9 


42 


72.4 


No 


3 


5.5 


4 


6.9 


Not Applicable 


13 


23.6 


12 


20.7 


Total 


55 


100.1 


58 


100.0 



ERiC 
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Table 85 

Responses to User Questionnaire by Occupation Category 
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Table 86 

Responses to User Questionnaire by Occupation Category 

Question 6: Which of the following materials or equipment are most lacking 

in your work situation? 



Response 

Microfiche reader 
Microfiche reader-printer 
Microfiche duplicator 
Government Reports 

Adequate library facilities close by 
Total 



Administrators 


Non-Admi nistrators 


f 


% 


f 




25 


19.5 


26 


22-6 


28 


21.9 


21 


18.3 


31 


24.3 


25 


21 -7 


19 


14.8 


20 


17.4 


25 


19.5 


23 


2(M) 


128 


100.0 


115 


100.0 



O 

ERIC 



262 



252 



Table 87 

Responses to User Questionnaire by Occupation Category 



Question 7: Which, if any, of the following problems have you encountered in 

using the services of the Resources Center? 



Response 

Profile too general 

Too much information to wade through 

Information not pertiment to request 

Too little information 

Complete documents (hardcopy or 
microfiche) not available 

Microfiche reader not readily available 

Took too long to process request 

Confusion as to what kind of information 
the Center can provide 

No problems were encountered 

Other 

Total 



Administrators 


Non-Administrators 


f 


% 


f 


% 


17 


14.4 


11 


9.6 


T 

/ 


5.9 


6 


5.3 


13 


11.0 


19 


16.7 


6 


5.1 


15 


13.2 


11 


9.3 


5 


4.4 


16 


13.6 


9 


7.9 


15 


12.7 


16 


14.0 


7 


5.9 


9 


7.9 


20 


17.0 


21 


18.4 


6 


5.1 


3 


2.6 


1 1 3 


100.0 


114 


100.0 



O 

ERIC 



283 
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Taole 38 

Responses to User Questionnaire by Occupation Category 



Question 8: Rank the following suggestions for improving the Center's services by placing "l" beside the 
most needed, M 2 H beside the next most, etc., and "5" beside the least needed Improvement. 



Administrators 



Rank: 




I 




2 




3 




4 




5 


Mean 


Response 




% 


jf 


% 


_f 


% 




% 




% 


Rank 


More specific request forms 
(n=51 ) 


15 


29.4 


5 


9.8 


10 


19.6 


8 


15.7 


13 


25.5 


2.980 


More accurate computer search 
(n*46) 


4 


8.7 


9 


19.6 


5 


10.9 


15 


32.5 


13 


28.3 


3.522 


Shorten the processing time of 
requests (n=51) 


17 


33.4 


10 


19.6 


7 


13.7 


8 


15.7 


9 


17.6 


2. *47 


Provide more expanded subject 
area coverage (n=51) 


18 


35.3 


10 


19.6 


13 


25.5 


6 


11.8 


4 


7.8 


2.373 


Keep expanding and updating 


22 


38.6 


17 


29.8 


9 


15.8 


6 


10.5 


3 


5.3 


2.140 



the data base (n=57 



Rank: 




Non- 

1 


•Administrators 

2 


3 




4 




5 


Mean 


Response 


Jf 


% 


£ 


% 




% 


_f 


% 




% 


Rank 


More specific request forms 
in-53; 


10 


18.9 


7 


13.2 


12 


22.6 


9 


17.0 


15 


2 


3.226 


More accurate computer search 
(n*-49) 


8 


16.3 


1 


2.0 


8 


16.3 


17 


34.8 


15 


30.6 


3.612 


Shorten the processing time of 
requests (n=56) 


14 


25.0 


9 


16.1 


10 


17.8 


9 


16.1 


14 


25.0 


3.000 


Provide more expanded subject 
area coverage (n=58) 


16 


27.6 


31 


36.2 . 


K 


24.1 


4 


6.9 


3 


5.2 


2.259 


Keep expanding and updating 
the data base (n=57) 


28 


49.0 


14 


24.6 


3. 


E.3 


7 


12.3 


5 


8.8 


2.070 
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Table 89 

Responses to User Questionnaire by Occupation Category 
Attitude Scale 



Administrators 









No. 


of Responses 




Mean 




Question 


SA(1> 


mi 


N( 3) 


mi 


ISM 


Score 


10. 


The Retrieval Center has bee>i a real help 
to me * 


30 


32 


10 


6 


1 


1.937 


12. 


The Retrie' .cer should be continued 

and expanded 


47 


25 


5 


1 


1 


1 .532 


14. 


The idea behind the Retrieval Center is 
a good one 


55 


16 


4 


2 


0 


1.390 


15. 


Children are benefiting because of the 
existence of the Retrieval Center 


saM 


33 

Mil 


22 

N(3) 


1 

0(2) 


1 

SD(1) 


2.077 








11. 


1 feel the Federal Government is not justified 
in supporting the cost of the Retrieval Center. 


1 


5 


6 


21 


46 


1.658 


13. 


The Retrieval is of questionable value .... 


0 


5 


5 


25 


43 


1.641 


Mean Total Attitude Score for Administrators: 
Standard Deviation 


10.224 

3.789 














Non-Administrators 
SA(1 ) 


MU 


MM 


D(4) 


SD(5) 


Mean 

Score 


10. 


The Retrieval Center has been a real help 
to me 


23 


27 


13 


4 


4 


2.141 


12. 


The Retrieval Center should be continued 
and expanded 


42 


21 


6 


2 


2 


1.644 


14. 


The idea behind the Retrieval Center i? 
a good one 


51 


17 


2 


1 


1 


1.389 


15. 


Children are benefiting because of the 
existence of the Retrieval Center 


14 


19 

Mil 

3 


31 

Mil 

12 


5 

D(2) 

26 


2 

SDil). 

28 


2.465 


11. 


SA(b; 

I feel the Federal Government is not justified 
in supporting the cost of the Retrieval Center. 3 


1 .906 


13. 


The Retrieval is of questionabl value, r . . 


. 1 


5 


9 


23 


34 


1.833 



Mean Total Attitude Score for Non-Admi nis trato^s : 
Standard Deviation 




11.536 

4.312 



Responses to User Questionnaire by Years Professional Experience 
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Appendix F 



Verbatim Responses 

to Open Categories on 
The User Questionnaire 
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User Questionnaire 

List of "Other" Responses on Question #1 

Please check the main purpose(s) for which you used the services of the 
Resources Center. 

-Information for a workshop 
-Evaluation of drug abuse program (CDE) 

Dissertation 

-Developing an R&D conceptual memo 

-To have print-out on reading, available for students at the Denver Center 

-Bibliography from specific educator 

-Education Assoc, results with sanctions 

-Provide USOE staff material 

-Secondary curriculum for slow learners 

-Preparation of federal proposals (2) 

-At invitation of director 
-College visited the Resource Center 
-Methods of Management of Instructional data 
-0E Staff Studies 

-Evaluation of Filmstrips of science from different companies 

-Dissertation Bibliography 

-Gathering facts on playground surfacing 
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User Questionnaire 

List of "Other" Responses on Question #2 
Where or how did you first hear about the services of the Resource Center? 

-Instructor 

-Husband 

-Presentation by Greeley personnel 
-Through library (5) 

-personal contact (3) 

-on library shelf 
-Student 

-State Education Department (5) 

-SD Title III Personnel 

-Report of State Dept of Education personnel at library conference 
-State Resource Personnel 
-Through a proposal for funding 
-USOE 

-Have known about it from start 
-Jim Simpson 

-Western Nevada Regional Education Center 
-Graduate School 
-College professor (2) 

-Utah School 3d. 

-ERIC parallel 

-Proposal 

-SIRS 

-Insti tute 
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User Questionnaire 



List of "Other" Responses on Question #7 



Which, if any, of the following problems have you encountered in using 
the Services of the Resources Center? 

-Manual search was not as thorough as it could have been 

-Microfiche difficult on^es 

-Some minor gaps in completeness of reports 

-Little I did not already know 

-Reports varied 

-Not sure of source of materials 
-Didn't receive any of the documents I ordered 
-Much of material too dated 
-Great help, but took two months 

-Microfiche reader only available at resource center 
-too little time (mine' * it justice 
-All materials not 
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List of "Other" Responses on Question #8 
Other suggestions for improving the Center's services than those listed. 

-Included in expanded subject area coverage, include specific literary works 
and reviews 

-They do "good" work 

-We felt the materials were excellent 

-Don't send out microfiche unless a reader is available 

-Provide material, not theory--we ' re full of that 

-Results need analysis and synthesis 

-Continue the program 

-Delineate between information on a speech or research study. Would 
like to know what types of data go into the computer. Thought it was 
completed research only. 

-Materials seemed quite up to date 

-Services do not appear too well publicized 

-Mission-oriented analysis papers aimed at particular practitioner audiences 

-Technical assistan 3 to clients to help them interpret and use the 
information received. 

-Do some in-depth relevance studies 

-Workshops to train systems operators and user to use information 

-Each school district might be given a workshop on use and availability 
of ERIC material 

-It may be well to include an educational sheet with each order suggesting 
to users how to make their requests more specific and thereby more useful. 

-Better dissemination of information about services 

-Didn't furnish the manual search material that they promised 

-Improve the evaluation form 

-I feel any help given has not justified the amount of time consumed in 
reports, materials to go through, etc. So far, I have really received 
no help at all. 

-Request filled promptly but microfiche films not pertinent. 

279 



269 



User Questionnaire 

Responses to Question #9 

Car. you cite any instances where information from the Resources Center has 
contributed to changes in programs and practices? 




-Special education 

-Material provided basis for change in social studies program. Present 
projects include salary schedule modifications, guidance program planning 
and development of humanities program at the secondary level. 

-Open-space instruction; individualized reading program 

-Planning to purchase 2 tachistoscopes for phase training 

-Curriculum revision for new building; provided information for own 

profession and personal growth of knowledge in areas related to my major. 

-Development of secondary reading program 

-Not directly, although the Resource Center provided two of our courses 
with a specific program. 

-Open space; individualized instruction in reading 

-Have better and more thorough information about own discipline 

-Changing of staffing patterns 

-Flexible scheduling in high school 

-Curriculum revision 

-Helped in providing more 'inquiry-centered' BSCS biology cl for students 

-Help in building a continuum of skills for a K-12 English language arts 
course of study 

-Plan to revise curriculum on basis of materials received 
-Thesis research 
-Remedial reading (2-6) 

-Area is curriculum, but too little time to evaluate changes 

-Development of team teaching in new junior high school 

-May use for remedial reading and environmental education information 
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User Questionnaire 

Responses to Question #9, continued 

-Students used it for enrichment (to accompany a reading of the Iliad) 

-Could eventually influence areas of instruction 

-My own research is for program learning ideas. The research that has been 
done in this area and hopefully for curriculum revision 

-An aid to programmed math for special students 

-Material being used to help gain background to discuss policy change with 
school board 

-Yes 

-Yes, in new program of re-grouping for reading; in attempt to set up 
individualized learning program 

-Possibly, can't give direct examples 

-Yes: 1. World of Work program 

2. Study on attitudes 

3. Information on Dyslexia 

4. Drug education material 

-Helps me develop curriculum (Teacher — college or univ) 

-Valuable suggestions 

-Only in my own personal teaching 

-Open space, individualized reading 

-We are evaluating our secondary handicapped and counsel irg program 
because of information we have received 

-Assisted in development of rights, responsibilities, regulations 
student handbook for the district 

-Has aided in program decisions as to kinds of centers to be developed 
and activities they might participate in. 

-In process of curriculum revision now as a partial result of material 
obtained 

-Helped us establish a pass-fail grading system for some of our classes 

-Information from the Center used for direction or guidance in Special 
Education services 
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User Questionnaire 

Responses to Question #9, continued 



•In my own case, 
helping me and 



I have received information about organizing IMCs which is 
my school's faculty to plan for our new IMG 9 months hence. 



Information about individualized instruction 
in effecting some curriculum changes. 



has assisted our principal 



-General background received from such research merely helps keep me a bre ^t 

of changing philosophies and practices that help me deal with everyday problems. 



-Some articles about noise are being used to design a program for use in a 
"total communication" unit to be presentee to elementary school children. 

iThe materials sent on the New York State K-12 English curriculum has helped 
a great deal in our working on a continuum of Language Arts skills. 



-Back-up information on a negotiating problem. 

-In training situations, this service has greatly effected the up-dating 
of programs and instruction. 

-Has helped supply information for changing reading program in our school- 
helped teachers to prepare for shift. 

-In some of my presentations in composition. 



-1 . Mesa Elementary School, BVPS 

2. Boulder Valley School Board 

3. CSU - Doctoral program in education 



No. of Responses: 



Yes 


48 


28.6% 


No 


58 


34.5% 


No Response 


62 


36.9% 


No. of programs 


named - 


62 
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